Example: Rubric for an essay
The holistic critical thinking scoring rubric, used in conjunction with fixed-choice tests, can provide multiple measures of critical thinking performance. As with all student assessment devices, care should be taken to assure interrater reliability in the ratings being generated by instructors.
Permission is granted to reproduce copies of this rubric for teaching assessment, research by students, faculty, and administrators at public or non-profit educational institutions.
Instructions for using the rubric
1. Understand the construct.
This four level rubric treats critical thinking as a set of cognitive skills supported by certain personal dispositions. A good critical thinker engages in analysis, interpretation, evaluation, inference, explanation, and meta-cognitive self-regulation. The ability to fairly pursue the reasons and evidence wherever they lead is crucial to reaching sound decisions and resolving complex problems.
2. Differentiate and focus
Holistic scoring requires focus. In any essay, many elements must come together for overall success: critical thinking, content knowledge, and technical skill. Deficits or strengths in any of these can draw the attention of the rater. However, in scoring for any one of the three, one must attempt to focus the evaluation on that element to the exclusion of the other two.
- Ideally, in a training session with other raters one will examine sample essays (videotaped presentations, etc.) which exemplify each of the four levels. Without prior knowledge of their level, raters should evaluate and assign ratings to these samples. After comparing these preliminary ratings, collaborative analysis with the other raters and the trainer is used to achieve consistency of expectations among those who will rate the actual cases. Training, practice, and inter-rater reliability are the keys to a high quality student assessment.
- Usually, two raters will evaluate each essay/assignment/project/performance. If they disagree there are two ways to resolve the discrepancy: by mutual conversation between the two raters, or by using an independent third rater. Discrepancies between raters of more than one level suggest that detailed conversations about the critical thinking construct and about project expectations are in order. This rubric is a four level scale; half point scoring is inconsistent with its intent and conceptual structure. Further, at this point in its history, the art and science of holistic critical thinking evaluation cannot justify asserting half-level differentiations.
- If working alone, or without samples, one can achieve a greater level of internal consistency by reviewing and giving essays preliminary ratings. Before long, clusters of similar quality will develop. At that point, you can more confidently assign a firmer critical thinking score using this rubric. After assigning preliminary ratings, a review of the entire set assures greater internal consistency and fairness in the final ratings.
4 Consistently does all or almost all of the following:
Accurately interprets evidence, statements, graphics, questions, etc. Identifies the salient arguments (reasons and claims) pro and con. Thoughtfully analyzes and evaluates major alternative points of view. Draws warranted, judicious, non-fallacious conclusions. Justifies key results and procedures, explains assumptions and reasons. Fair-mindedly follows where evidence and reasons lead.
3 Does most or many of the following:
Accurately interprets evidence, statements, graphics, questions, etc. Identifies relevant arguments (reasons and claims) pro and con. Offers analyses and evaluations of obvious alternative points of view. Draws warranted, non-fallacious conclusions. Justifies some results or procedures, explains reasons. Fair-mindedly follows where evidence and reasons lead.
2 Does most or many of the following:
Misinterprets evidence, statements, graphics, questions, etc. Fails to identify strong, relevant counter-arguments. Ignores or superficially evaluates obvious alternative points of view. Draws unwarranted or fallacious conclusions. Justifies few results or procedures, seldom explains reasons. Regardless of the evidence or reasons, maintains or defends views based on self-interest or preconceptions.
1 Consistently does all or almost all of the following:
Offers biased interpretations of evidence, statements, graphics, questions, information, or the points of view of others. Fails to identify or hastily dismisses strong, relevant counter-arguments. Ignores or superficially evaluates obvious alternative points of view. Argues using fallacious or irrelevant reasons, and unwarranted claims. Does not justify results or procedures, nor explain reasons. Regardless of the evidence or reasons, maintains or defends views based on self-interest or preconceptions. Exhibits close-mindedness or hostility to reason.
Holistic critical thinking scoring rubric. Retrieved May 3, 2004 from California Academic Press Web site: http://www.uog.edu/coe/ed451/tHEORY/HolisticCTrubric.pdf
Facione, P. A. (1990). Critical thinking: A statement of expert consensus for purposes of educational assessment and instruction: Research findings and recommendations. American Philosophical Association. ERIC Document Reproduction Service No. ED 315 423.
Facione, P. A., & Facione, N. C. (1994). The holistic critical thinking scoring rubric. Millbrae, CA: The California Academic Press.