CRITICAL, CREATIVE, REFLECTIVE AND LOGICAL THINKING IN THE NEMP ASSESSMENTS
10. SUMMARY
10.1 INTRODUCTION

The research involved considering all of the tasks in the 1995 - 2003 NEMP assessments to identify those which had the potential to assess critical, creative, reflective or logical thinking. A total of 711 tasks were considered and 159 of these involved one, or more, of these kinds of thinking.

Each of the kinds of thinking were considered separately and the tasks involved are presented and discussed in sections 4 - 7.

The marking criteria for the tasks were then considered to see to what extent the potential to evaluate the different kinds of thinking was realised in the assessments. The nature of these criteria is reflected well in the way the results of the assessment are reported in the subject reports. Consequently, it was decided that it was not necessary to examine the marking criteria for all of the 159 tasks identified. A sample of 52 tasks, selected to represent each type of thinking and a spread across the curriculum was chosen.

The nature of these marking criteria and the extent to which they capture the nature of the thinking involved is also presented in sections 4 - 7.

Most of the marking, and reporting, criteria, while assessing the results of critical, creative, reflective or logical thinking, did not capture the nature of the thinking itself. Consequently, tasks which had the greatest potential for identifying the nature of the thinking were identified using criteria suggested by Halpern (2003, page 361). In the context of the NEMP assessments, this meant using tasks which:

   
     
•
are in a one-to-one interview format
• are open-ended
• ask for explanations or justifications
         

These tasks are referred to in this report as Halpern tasks. 29 such tasks were identified. The relationship of these tasks to the different kinds of thinking and to the curriculum areas is discussed in section 8

Finally, the question as to whether further research into the video tapes of the responses to some of the Halpern tasks would be useful was considered.
         
10.2 THE THINKING TASKS

Of the 711 tasks considered, 159(22%) were judged to have the potential to assess one, or more of the thinking skills. 3% involved critical thinking, 6% creative thinking, 8% reflective thinking, and 6% logical thinking.

The assessments for Maori students (1999 – 2002) used a selection of tasks from the general assessments. The distribution of thinking tasks is similar to that of the general assessment. There were 164 tasks of which 31(19%) were thinking tasks, 5% critical thinking, 3% creative thinking, 7% reflective thinking, and 4% logical thinking.
         
10.3 CRITICAL THINKING
The working definition of critical thinking in this report is that it is:
  Thinking which involves evaluation and, perhaps, challenge.
Tasks involving critical thinking were found in 7 of the 12 curriculum assessment areas with Art, Technology, and Listening and Viewing contributing the most tasks. Most of the tasks require students to explain, justify or discuss their responses. The obvious potential which this presents for assessing the thinking of students was realised very well in some, but not all marking criteria.
         
10.4 CREATIVE THINKING
The working definition of creative thinking in this report is that it is:
  Thinking which is directed towards solving a problem in one’s own way. It often involves imagination and initiative.
Tasks involving creative thinking were found in 7 or the 12 curriculum assessment areas with Art, Music, Reading and Speaking, and Writing contributing the most tasks. All of the tasks involve producing something which is the result of creative thinking. None of the tasks require students to explain or discuss the processes used. Consequently, the marking criteria assess the quality of the product of the thinking rather than the thinking itself.
         
10.5 REFLECTIVE THINKING
The working definition of reflective thinking in this report is that it is:
  Thinking which involves looking back on one’s previous thinking, knowledge, and understanding.
Reflective thinking tasks were found in 11 of the 12 curriculum assessment areas. Health and Physical Education contributed the most tasks with the rest spread widely over the other curriculum areas. The general nature of reflective thinking means there are a very wide range of questions and contexts. In many of the tasks students are asked to “think about ….”. The marking criteria seemed to capture the nature of the students thinking quite well.
         
10.6 LOGICAL THINKING
The working definition of logical thinking in this report is that it is:
  Thinking which is directed towards making deductions or presenting arguments.
Logical thinking tasks were found in 8 of the 12 curriculum assessment areas. Science, Mathematics, and Technology contributed the most tasks. Classifying, planning and explaining were the focus of many of the tasks. The tasks tended not to be open ended which reduced the potential for the marking criteria to capture the thinking behind the responses.
         
10.7 THE HALPERN TASKS
The Halpern tasks, which had the greatest potential for assessing the thinking of students, were those which:
•
 are in a one-to-one interview format
•
 are open-ended
•
 ask for explanations or justifications
   
10.7.1 The distribution of Halpern tasks
29 such tasks were identified, 14 involved critical thinking, 11 reflective thinking and 4 logical thinking. None of the creative tasks satisfied the criteria.

In the creative tasks only 5 of the 40 tasks used a one-to-one format and the students were not asked to explain or justify their responses in any of these. There was a tendency for the logical tasks not to be open-ended.

It was felt that the potential to assess the thinking of students was realised to a greater or lesser extent in 21 of the 29 Halpern tasks.

There were marked differences in the distribution of Halpern tasks across the curriculum assessment areas with 7 of the 12 areas contributing none, or only one task. The tasks in a number of assessment areas tend not to be open-ended, and the more creative curriculum areas tended not to use the one-to-one task approach.

The curriculum area which was most successful in assessing student thinking was Art.
   
10.8 TASKS FOR FUTURE RESEARCH
There did seem to be a number of critical and reflective thinking tasks among the Halpern tasks which might warrant further research, based on the evidence of the video tapes of the students completing the task, into the nature of the thinking involved.
   
11. CONCLUSIONS
Each of the following conclusions should be read remembering that the NEMP assessments were not principally designed to assess the nature of the thinking of the students.
1. There is a good distribution of tasks involving critical, creative, reflective and logical thinking in the NEMP assessments. The thinking is tested in a very wide range of contexts across the curriculum.
2. It might be worth publishing a list of the best thinking tasks in each category as a resource for teachers.
3. The critical thinking tasks were, perhaps, the most successful in assessing the thinking of students.
4. The creative thinking tasks were very successful in assessing the results of creative thinking, but did not explore the nature of the thinking involved.
5. The reflective thinking tasks seemed to have more potential for assessing student thinking than was realised.
6. The logical thinking tasks tended not to be open-ended which reduced their effectiveness as thinking assessment tools.
7. The marking schemes varied greatly in their effectiveness in capturing the thinking skills of the students.
8. The tasks which showed the most potential for assessing thinking skills were those which:
 

• are in a one-to-one interview format
 

• are open-ended
 

• ask for explanations or justifications
  and for which the marking criteria look at the nature, as well as the result of the thinking by probing the reasons behind a response.
9. Consideration should be given to trying to include some tasks of this nature in all NEMP assessments.
10. Teachers should be encouraged to use tasks of this nature at all levels of teaching and in all curriculum areas.
11. It does seem likely that the video tapes of the student responses to some of the NEMP assessment tasks contains very useful information, not captured by the marking criteria, concerning the nature of the thinking involved. Further research in this area might be very valuable.
   
12. REFERENCES
12.1 NEMP REPORTS
  National Education Monitoring Report: Assessment Results. Dunedin: EARU, University of Otago
 
1. Science 1995
2. Art 1995
3. Graphs, Tables and Maps 1995
4. Music 1996
5. Technology 1996
6. Reading and Speaking 1996
7. Information Skills 1997
8. Social Studies 1997
9. Mathematics 1997
10. Listening and Viewing 1998
11. Health and Physical Education 1998
12. Writing 1998
13. Science 1999
14. Art 1999
15. Graphs, Tables and Maps 1999
16. Assessment Results for Maori Students, Science; Art; Graphs, Tables and Maps 1999
17. Music 2000
18. Aspects of Technology 2000
19. Reading and Speaking 2000
20. Assessment Results for Maori Students, Music; Aspects of Technology; Reading And Speaking 2000
21. Information Skills 2001
22. Social Studies 2001
23. Mathematics 2001
24. Assessment results for Maori Students, Information Skills, Social Studies; Mathematics 2001
25. Listening and Viewing 2002
26. Health and Physical Education 2002
27. Writing 2002
28. Assessment Results for Maori Students, Listening and Viewing; Health and Physical Education; Writing 2002
29. Science 2003
30. Visual Arts 2003
31. Graphs, Tables and Maps 2003
   
12.2 OTHER REFERENCES
  Fasko, Daniel, Jr. (ed). (2003). Critical thinking and reasoning. Current research, theory, and practice. Hampton Press Inc., Cresskill NJ.
  Halpern, Diane F. (2003). The “how” and “why” of critical thinking assessment. In Fasko (2003) pp 355 – 366.
  Knight,G. (2002). Essential Skills: Examination of Essential Skills performance. A National Education Monitoring Project Probe Study Report. Dunedin: EARU, University of Otago.
  Ministry of Education (1993). The New Zealand Curriculum Framework. Wellington: Ministry of Education.
  Ruggiero, V.R. (2004). The art of thinking. New York: Pearson Education Inc.
   

prev page
top of page    |    return to Probe Studies - INDEX   |    return to Other Studies menu
For further information and contact details for the Author    |    Contact USEE