ENSURING
CONSISTENCY: THE NEMP PROCESS OF CROSS-MARKING
Nicole Brown
This
report describes and reviews the cross-marking process developed
by NEMP to score performance-based assessment tasks on a consistent
basis. Cross-marking uses facilitated discussion on the part of
markers to establish, apply and verify marking criteria. Random
selections are made of the work of up to six students who have
completed each set of NEMP tasks. All NEMP markers then grade
this work at strategic points during the marking process so as
to build and check consistency. |
| A
group of markers independently graded one student performance, using
the criteria given on the marking schedule. Markers were then asked,
in a group session, to explain their scores. The goal was to clarify
the choices and decision-making processes markers used and to obtain
agreement on the grades to be assigned. The marking of each task involved
three sessions—the beginning, middle and end of the marking.
Two student performances were considered in each cross-marking session.
To gauge the consistency of scoring overall, |
markers were asked not to change grades they assigned during the cross-marking sessions. All cross-marking grades were collated into a grid showing the pattern of scores awarded by each marker, then averaged to provide a final score for each ‘cross–marked’ student script. The cross-marking process was then observed with a view to identifying those factors that hindered or facilitated this process. The extent to which this process met consistency objectives was also considered. |
| The
marking process • Collaboration and joint ownership help develop good performance assessment rubrics, and are of more value if they occur during the marking process. • Markers need to share their collective experience of students’ responses during scoring and use it to adjust the rubrics where appropriate. Some rubrics designed using collaborative processes before the actual marking period were difficult to apply when used by a different group of markers during the actual marking period. • An active facilitative approach is necessary. Markers may be uncomfortable about explaining their scoring decisions, so positive and sensitive facilitation skills are needed to encourage, acknowledge and consider different points of view. Once markers are comfortable about contributing to discussion, the facilitation process can focus on identifying key issues and resolving them through the flexibility of the marking schedule. |
Consistency • As tasks require greater professional judgement, the consistency of scoring decreases. • Consistency in scoring is generally greatest when a student’s performance is considered particularly strong. • Inconsistencies in scoring a student performance spreading across three different scores seemed to arise from most markers having a similar view of the performance but coming down on either side of a marking ‘boundary’. • Teachers who awarded scores falling significantly outside the common range of agreement appeared to be influenced by whether they had independently marked a series of very good or very poor performances just before the cross-marking took place. |
| Collaborative discussions about the cross-marking process appear to have a strong influence on its validity and reliability. The essential principle of treating the scoring of student performance as a discussion process involving collaboration, joint ownership and facilitation is one that can easily be replicated in school settings. |  |