Thomas, Tagg and Holton (2003, p1) write:

“Two powerful tools in guiding the development of assessment items are a content Framework and a Curriculum Map. A content Framework classifies the content of a subject into major categories or big ideas and sub-divides the big ideas into major or key themes. What is being taught in classrooms ought to relate to children learning these major ideas and related themes. Furthermore, a curriculum will specify a multiplicity of achievement objectives across a range of difficulty levels that need to be taught and mastered as the substance of the major curricular themes and ideas. A curriculum map relates the detailed achievement objectives to the major categories and key themes identified in the content Framework.”

In the asTTle Mathematics Content Framework (Levels 2-6), described by Thomas, Tagg and Holton (p5), there are four content strands subdivided into eight major categories. These are further subdivided into key themes.

In the associated Curriculum Map all of the achievement objectives in the mathematics curriculum (Ministry of Education, 1992) are related to the major categories and key themes of the content Framework.

The notation which is used to classify the achievement objectives indicates:

So N3-5 is the fifth listed level three achievement objective in the number strand of the curriculum document. It is:

“recall the basic multiplication facts”

The map also makes reference to some missing achievement objectives and to mathematics achievement standards for NCEA Level 1.

The concepts and notation of the asTTle Curriculum Map proved particularly useful in this project. It was possible to assign almost all of the task components in the NEMP assessments to curriculum achievement objectives and this gave a very good indication of the curriculum coverage of the assessments.
Each of the NEMP reports (eg. NEMP, 2002) contain the following statement:

“The assessment tasks emphasize aspects of the curriculum which are particularly important to life in our community, and which are likely to be of enduring importance to students. Care is taken to achieve balanced coverage of important skills, knowledge and understandings within the various curriculum strands, but without attempting to slavishly follow the finer detail of current curriculum statements. Such details change from time to time, whereas national monitoring needs to take a long-term perspective if it is to achieve its goals.”

The asTTle classification should enable us to make it clear which aspects of the curriculum are being emphasised in the NEMP assessments.

As with most classification systems, there are some tasks for which the classification is obvious, some which could fit in to more than one category, and some which do not fit clearly into any category at all. This can be illustrated by some of the number computation tasks.

1. The 1997 item Multiplication Facts, in which students were asked to attempt a series of 30 multiplications presented at 5 second intervals, clearly assesses the fifth objective listed in Level 3 of the Number strand of the curriculum:
   

   N3-5	Recall the multiplication facts and is classified accordingly.

2. The 1997 Fractions task is more problematic. The task contains addition, subtraction, multiplication and division questions involving fractions and mixed numbers.
   
   The achievement objectives relating to fractions are:
   

   N2-5	Write and solve story problems which involve halves, quarters, thirds and fifths.
   N3-7	Solve practical problems which require finding fractions of whole number and decimal amounts.
   N4-3	Find fractions equivalent to one given.
   N4-4	Express a fraction as a decimal, and vice versa.
   N4-6	Express quantities as fractions or percentages of a whole.
   N4-9	Find a given fraction or percentage of a quantity.
   N6-1	Perform basic operations on fractions and mixed numbers.

   
   The only achievement objective which precisely fits the components of the Fractions task is N6-1 and most of the components clearly belong there. However, some of the simplest components such as: 1 - 1/2 do not involve computational skills, but just an understanding of the relationship between a half and a unit. This seems to be a level 2 skill. This opinion is reinforced by the level 2 ‘suggested learning experiences’ table in the curriculum which includes:
   
    exploring fractions (halves, quarters, thirds, fifths)
   
   Consequently, components of this nature were classified N2 without an achievement objective number.
   
3. The 2001 Division Facts task contains 30 items such as 32 ÷ 8.
   
   Unlike the other three arithmetic operations, the recall of basic division facts does not appear as an achievement objective. The only achievement objectives involving whole number division are:
   
   

   N2-10	Write and solve story problems which involve whole numbers, using addition, subtraction, multiplication and division.
   N2-11	Write and solve story problems which require a choice of any combination of the four arithmetic operations

   
   However, it seems reasonable to assume that the basic division facts are associated with the corresponding multiplication facts which appear as a level 3 objective.
   
   Consequently, the Division Facts task was classified N3, again without an achievement objective number.
   
   
4. There were two 2001 tasks, Farmyard Race and Photo Line-Up, to which the researcher was unable to associate a curriculum level. They are logic items which belong to the Mathematical Processes strand of the curriculum. The achievement objectives for this strand are not associated with specific levels of the curriculum.
   
   

There is clearly very wide coverage of the level 2, 3 and 4 number achievement objectives. Of those that are missing almost all are covered by later objectives. For example:

OBJECTIVE	No. OF TASKS	OBJECTIVE	No. OF TASKS
M2-1	6	M4-3	2
M2-2	5	M4-5	1
M2-3	2	M4	1
M2-4	2
M3-1	8
M3-2	5
M3-3	1

There is again a wide coverage of the level 2 and 3 objectives with the level 4 being a little weaker.

It is useful to examine the curriculum levels of the tasks given to year 4 and year 8 students. The table below indicates the number of tasks, from both the 1997 and 2001 assessments, which involved achievement objectives from the given curriculum level. The 1997 Money tasks have been included with the measurement tasks.

	YEAR 4					YEAR 8
	L1	L2	L3	L4		L1	L2	L3	L4	L5	L6
Number	2	23	11	2		1	19	15	10	1	2
Measurement	0	16	16	1		0	12	18	5	0	0
Geometry	0	10	2	0		0	10	5	1	0	0
Algebra/Stats	0	8	5	2		0	6	8	4	0	0
Total	2	58	31	4		1	47	46	20	1	2
%	2	61	33	4		1	40	39	17	1	2

The diagram below, taken from the mathematics curriculum document (Ministry of Education, 1992, p17), indicates:

“The general relationship between the levels and years of schooling…”

(click on figure to enlarge)

From this diagram one might expect that the majority of year 4 students would be working around the level 2 achievement objectives. This is consistent with the 61% of the NEMP tasks which involved these objectives. There were 37% of tasks requiring higher level objectives. The mean level of the achievement objectives for the year 4 tasks was 2.4.

The diagram also suggests that the majority of year 8 students would be working on level 4 objectives. However, only 17% of their tasks were at this level and only 3% at a higher level. The mean level of the achievement objectives for the year 8 tasks was 2.8.

This obviously arises from the NEMP decision to include a large number of tasks which were attempted by both year 4 and year 8 students. In the 2001 assessment (NEMP, 2002, p11), of the 101 tasks administered, 43 were identical for year 4 and year 8 and a further 28 contained some common components for both years.

The purpose of these common tasks is to provide a measure of growth from year 4 to year 8. However, this growth has been difficult to interpret (Knight, 2001, p7) and the data above does seem to indicate that the emphasis on it has distorted the monitoring of the performance of year 8 students.

next page