home
Assess teaching

Product analysis conclusions

Evaluate your results based on how well they answer your central questions and base conclusions on the objectives you set for your analysis. A product analysis can tell you whether learning is going as intended. In the example below, an instructor notices from the analysis of Essay 1 that nearly all students are meeting objective 2 but almost no students are meeting objectives 3 and 4 and a disappointing percentage are meeting objective 1. The instructor might conclude that she needs to rewrite the directions for Essay 1 to emphasize the requirements for originality and providing persuasive arguments and that she should spend more class time having students apply key concepts. Conducting a document analysis and peer observation would help verify these conclusions.

Establishing what performance standards are acceptable prior to conducting a product analysis is important when making conclusions. For example, the table below shows that over 90% of students are meeting objective 2 for each assignment. Whether this is acceptable depends on the instructor's standard. If the instructor has set the standard that all students (100%) will be able to use examples of key concepts correctly then he would conclude that he needs to make adjustments to his instruction or assignments.

Setting the priority of each objective will also guide conclusions. You may set a lower standard of success for an objective of medium priority compared to one of high priority. For example, you might view 90% of students meeting a high priority objective as successful and view 70% of students meeting a medium priority objective as successful.

By organizing information in this way it is easier to formulate possible explanations for the results and identify solutions. Are learning objectives not being communicated clearly? Is the material too challenging? Are students capable of doing the work but need more information or resources? A follow-up survey could help answer such questions, and analyses of the syllabus, lecture notes, and handouts (document analysis) could reveal whether you are clearly communicating objectives and needed information.

Example

Percentage of students meeting objectives for writing assignments in HIS273, Section K.

Learning objective Priority Essay 1 Essay 2 Term paper

1. Application of concepts

high

65%

88%

75%

2. Use of examples

high

92%

95%

90%

3. Persuasive arguments

medium

20%

75%

82%

4. Originality

medium

10%

27%

38%

Additional information:

Patton, M.Q. (1990). Qualitative evaluation and research methods, 2nd ed. Newbury Park, CA: Sage.

Sweet, D. (1993) Student Portfolios: Classroom Uses. U. S. Department of Education Consumer Guide, Number 8, Archived material. Retrieved  June 21, 2006 from the Office of Educational Research and Improvement (OERI) of the U.S. Department of Education Web site: http://www.ed.gov/pubs/OR/ConsumerGuides/index.html

Page last updated: Sep 21 2011
Copyright © 2007, The University of Texas at Austin