A study of changes in mathematics achievement using data from
the first and second cycles of NEMP assessment
 
   

4. THE CLASSIFICATION OF TASK COMPONENTS BY PERFORMANCE EXPECTATION

4.1
INTRODUCTION


The second important feature of an assessment task, alongside its subject content, is the nature of the skill required for the completion of the task. In the Third International Mathematics and Science Study (TIMSS) a classification of such skills, called performance expectations, was used. This classification was used in this study.

The performance expectation categories are (Garden, 1997, p83):

“…non-hierarchical cognitive behaviours students are expected to be able to demonstrate as an outcome of their mathematics education.”

They are described as:

  1. Knowing - demonstrating familiarity with non-verbal mathematical representation, recall of mathematical objects and properties, recognition of mathematical content, and correct use of mathematical vocabulary and notation.
  2. Routine procedures - using equipment, performing routine procedures such as computing, graphing, measuring; or producing new information from old using an algorithmic approach using only a few steps.
  3. Complex procedures - performances where procedures are not well defined or where an algorithm cannot be used. It includes performances such as estimating, organising and displaying data, comparing mathematical objects, and classifying objects.
  4. Solving Problems - investigations in which the task is primarily to “solve the problem”. This expectation does not imply only the mastering of problems usually referred to in the literature as “non-routine”, and is applied to items which require at least two steps to obtain a correct response. However, it does include formulating and clarifying problems, developing solution strategies, predicting the results of operations before they are performed, and verifying the appropriateness of a solution to a problem.
  5. Justifying and proving - encompassing all aspects of mathematical reasoning including developing new notation, vocabulary, and algorithms; generalising and conjecturing; and justifying and proving.
  6. Communicating - relating mathematical representations, describing and discussing methodology and observations, and critiquing an idea or solution.

Each of the components of the tasks in the 1997 and 2001 assessments was classified according to the performance objective judged to be necessary for its completion.

4.2		 EXAMPLES OF CLASSIFICATION OF TASKS BY PERFORMANCE EXPECTATION The only difficulty encountered in classifying the components of tasks by performance expectation related to the distinction between complex procedures and solving problems. The examples given, such as ‘estimating’ as a complex procedure, and ‘developing strategies’ as solving problems, were helpful. However, the complex procedure criterion of “performances where procedures are not well defined or where an algorithm cannot be used” seemed, to the researcher, to apply to solving problems as well. Similarly, the solving problem criterion of “items which require at least two steps” applies to ‘organising and displaying data’ which is given as an example of complex procedure.

Perhaps the most important distinction is that complex procedures, although they are not well defined, are still procedural, while solving problems is a more investigative process.

The following examples will illustrate the decisions made in classification.
 

4.2.1 Knowing

Examples of are found in all the recall tasks involving the basic arithmetic operations:

w Addition facts   1997
w Multiplication facts 1997
w Subtraction facts 2001
w Division facts 2001

Other knowing tasks are:

w Video recorder    2001
  Students are asked to convert am and pm times to 24 hour clock times.
w Flat shapes (2)    2001
  Students are asked to name the shape of a label.

 
  4.2.2 Routine procedures

Examples are found in the arithmetic procedure tasks:

w Subtraction facts   1997
w Division 1997
w Addition examples 2001
w Multiplication examples  2001

Other routine procedure tasks are:

w Line of symmetry    2001
  Students are asked to draw the other side of an object using a given line of symmetry.
w Statistics items B    2001
  Students are asked to find the average (mean) age of a group of five children.


  4.2.3 Complex procedures

Some examples of complex procedure tasks are:

w Number and word problems (10) 1997
  Students are given the ingredients necessary to cook a meal for 12 people and asked to give those necessary for 6 people.
w Bean estimates 2001
  Students are asked to estimate the number of beans necessary to fit into trays of different sizes.
w Whetu’s frame 2001
  Students are asked to determine the quantity of pipe, and the number of corners, required to construct a frame shown in a diagram.
w Number patterns 1997
  Students are asked to write down the missing numbers in the patterns.


  4.2.4 Solving problems

Some examples of solving problems tasks are:

w Numbers in squares 1997
  A team task in which students have to place numbers in a square to give specified row, column and diagonal totals.
w Lump balance (4) 2001
  Students have to make a lump of plasticine which is one and a half times as heavy as a given lump.
w Cut it out 1997
  Students are asked to fold and cut a piece of paper so that it looks like an illustrated one.
w Farmyard Race 2001
  A team task in which students have to use a number of clues to place a set of plastic animals in order.


4.2.5 Justifying and proving

Understandably, there were few items involving this skill.

w Maths adviser   1997
  Students are asked help someone in their class by explaining the answers to questions.
w Number items (2)   1997
  Students are asked to justify their answer concerning the size of a bus needed for a school trip.


  4.2.6 Communicating

Again, there were relatively few items in this category. The tasks often involved both justifying and communication.

w Better buy (3) 2001
  Students were asked to explain why their solution was correct.
w Bank account 2001
  Students have to tell a story to explain what is happening with the money in a bank account. The information is provided in a graph.
w Statistics items B 2001
  Students are asked to explain to Maria why she is right or wrong in a statement she makes about computer use.



4.3		 PERFORMANCE EXPECTATIONS OF NEMP TASKS 

The table below gives the percentage of NEMP tasks in each area of knowledge which were judged to require the given TIMSS performance expectations. Most tasks required more than one performance expectation.

YEAR 4
YEAR 8
AREA
K
RP
CP
SP
JP
C
  
K
RP
CP
SP
JP
C
NUMBER
                          
1997
31
46
8
38
15
15
31
54
8
38
8
8
2001
39
44
39
22
0
0
32
47
42
21
0
5
Total
35
45
26
29
6
6
31
50
28
28
3
6
                           
MEASUREMENT                          
1997
0
57
57
14
0
7
0
57
50
50
0
14
2001
12
65
47
18
0
26
5
63
47
16
0
21
Total
6
61
52
16
0
16
3
61
48
30
0
18
                           
GEOMETRY                          
1997
20
40
0
60
0
0
17
33
0
67
0
0
2001
13
38
38
38
0
0
10
40
40
30
0
0
Total
15
38
23
46
0
0
13
38
25
44
0
0
                           
ALGEBRA/STATS                          
1997
0
33
100
33
0
0
0
17
100
33
0
17
2001
43
29
57
71
0
0
20
30
50
80
10
10
Total
23
31
77
54
0
0
13
25
69
63
6
13
                           
OVERALL                          
1997
13
47
39
32
5
8
13
46
36
46
3
10
2001
26
48
44
30
0
8
17
48
45
31
2
10
Total
20
48
42
31
2
8
15
47
46
32
2
10


The table indicates that, apart from justifying and proving, there is a wide coverage of the TIMMS performance expectations in the NEMP assessments. The coverage is very similar at year 4 and year 8, which is, perhaps, surprising. One might have expected a greater emphasis on the higher level skills at year 8.

next page
top of page    |    return to Probe Studies - INDEX   |    return to Probe Studies menu
  For further information and contact details for the Author    |    Contact USEE