Product analysis conclusions
Evaluate your results based on how well they answer your central questions and base conclusions on the objectives you set for your analysis. A product analysis can tell you whether learning is going as intended. In the example below, an instructor notices from the analysis of Essay 1 that nearly all students are meeting objective 2 but almost no students are meeting objectives 3 and 4 and a disappointing percentage are meeting objective 1. The instructor might conclude that she needs to rewrite the directions for Essay 1 to emphasize the requirements for originality and providing persuasive arguments and that she should spend more class time having students apply key concepts. Conducting a document analysis and peer observation would help verify these conclusions.
Establishing what performance standards are acceptable prior to conducting a product analysis is important when making conclusions. For example, the table below shows that over 90% of students are meeting objective 2 for each assignment. Whether this is acceptable depends on the instructor's standard. If the instructor has set the standard that all students (100%) will be able to use examples of key concepts correctly then he would conclude that he needs to make adjustments to his instruction or assignments.
Setting the priority of each objective will also guide conclusions. You may set a lower standard of success for an objective of medium priority compared to one of high priority. For example, you might view 90% of students meeting a high priority objective as successful and view 70% of students meeting a medium priority objective as successful.
By organizing information in this way it is easier to formulate possible explanations for the results and identify solutions. Are learning objectives not being communicated clearly? Is the material too challenging? Are students capable of doing the work but need more information or resources? A follow-up survey could help answer such questions, and analyses of the syllabus, lecture notes, and handouts (document analysis) could reveal whether you are clearly communicating objectives and needed information.
Example
Percentage of students meeting objectives for writing assignments in HIS273, Section K.
| Learning objective | Priority | Essay 1 | Essay 2 | Term paper |
|---|---|---|---|---|
1. Application of concepts |
high |
65% |
88% |
75% |
2. Use of examples |
high |
92% |
95% |
90% |
3. Persuasive arguments |
medium |
20% |
75% |
82% |
4. Originality |
medium |
10% |
27% |
38% |
Additional information:
Patton, M.Q. (1990). Qualitative evaluation and research methods, 2nd ed. Newbury Park, CA: Sage.
Sweet, D. (1993) Student Portfolios: Classroom Uses. U. S. Department of Education Consumer Guide, Number 8, Archived material. Retrieved June 21, 2006 from the Office of Educational Research and Improvement (OERI) of the U.S. Department of Education Web site: http://www.ed.gov/pubs/OR/ConsumerGuides/index.html
