home
Assess teaching

Analyzing products

Three useful approaches for analyzing student products are comparing student products with quality standards, writing narrative descriptions, and subjectively assessing quality.

Quality standards

To use quality standards, compare the content of student products with established learning objectives or standards. One approach is to record the number or percent of students who are meeting each learning objective for an assignment, or alternatively, to record the extent to which students are meeting objectives.

Use rubrics to get consistent scores across all students. If practical, use multiple evaluators who understand predetermined criteria and are trained in the evaluation procedure. [more]

Example

Rubric for analysis of writing assignment

Key

Not present = no use or demonstration of objective
Below criteria = little use or demonstration of objective or use is frequently inaccurate
Meets criteria = consistent use or demonstration of objective
Exceeds criteria = consistent and skillful use or demonstration of objective

Objective

Not present
(0 pts)

Below criteria
(1 pt)

Meets criteria
(2 pts)

Exceeds criteria
(3 pts)

Discussing concepts from readings

 

 

X

 

Applying concepts using specific examples

 

X

 

 

Developing an organized, persuasive argument

 

 

X

 

Presenting original ideas

 

 

 

X

To determine if a change in instruction has affected the quality of student products, you could compare the same product, such as a term paper, from the semesters before and after you made the change.

Narrative descriptions

Narrative descriptions clarify how products promote and demonstrate learning objectives, outlining the skills, concepts, and resources that students used to create the product. Focus on one to four core elements of the student product that best illustrate learning objectives. For example, you could focus on application of concepts, use of examples, persuasiveness of the argument, and originality. Use consistent terminology to make comparisons easier. For example, always use "good" to describe above average performance and "excellent" to describe superior performance.

Example

Narrative for analysis of writing assignment

Objective Narrative comments

Application of concepts

Applied five concepts correctly and one incorrectly. Applications were based on those presented in class but went beyond the basic idea. Good elaboration.

Use of examples

Provided detailed examples as part of each application. Some examples drawn from own experience while others derived from reading and lecture.

Persuasive arguments

Arguments were directly connected to application and examples but were only somewhat persuasive. Position not always stated clearly.

Originality

Some original thought and use of personal examples, but majority of content derived from course materials.

Combine narrative descriptions with quality standard comparisons for a more comprehensive product analysis.

Example

Rubric for analysis of writing assignment

Key:
0 = no use or demonstration of objective
1 = little use or demonstra tion of objective or use is frequently inaccurate
2 = consistent use or demonstration of objective
3 = consistent and skillful use or demonstration of objective

Objective Narrative Quality ratings

Application of concepts

Applied five concepts correctly and one incorrectly. Applications were based on those presented in class but went beyond the basic idea. Good elaboration.

2

Use of examples

Provided detailed examples as part of each application. Some examples drawn from own experience while others derived from reading and lecture.

3

Persuasive arguments

Arguments were directly connected to application and examples but were only somewhat persuasive. Position not always stated clearly.

1

Originality

Some original thought and use of personal examples, but majority of content derived from course materials.

1

Subjective assessments

Subjective assessments, which do not use pre-established standards to guide the analysis, are subject to bias and lack of focus.   However, subjective assessments may enable you to discover key aspects of the learning process that you might not have recognized ahead of time. Innovative students, for example, may develop original formats or concept applications that demonstrate learning but were not part of your original assignment objectives. Subjective assessments can also provide a more holistic assessment of quality compared to checklists or rubrics, although a lack of specificity makes it harder to identify common learning deficiencies or misunderstandings.

Additional information

Patton, M.Q. (1990). Qualitative evaluation and research methods, 2nd ed. Newbury Park, CA: Sage.

Sweet, D. (1993) Student Portfolios: Classroom Uses. U. S. Department of Education Consumer Guide, Number 8, Archived material. Retrieved June 21, 2006 from the Office of Educational Research and Improvement (OERI) of the U.S. Department of Education Web site: http://www.ed.gov/pubs/OR/ConsumerGuides/index.html

Page last updated: Sep 21 2011
Copyright © 2007, The University of Texas at Austin