Preparing for
Peer Observation

A Guidebook

 

Prepared by
The Center for Teaching Effectiveness
Main Building 2200
The University of Texas at Austin

Phone 512-471-1488

Marilla Svinicki, Director
Karron Lewis, Associate Director



(NOTE: A PDF version of this document is also available for printing.)



Contents:

 

 

Why peer evaluation and peer observation?

In a memorandum dated May 3, 1995, President Robert Berdahl asked deans and department chairs at the University of Texas at Austin for their recommendations for all changes in academic rank/status for 1996-97. An attachment to the memorandum, Guidelines for the Preparation of Supporting Materials and the Management of Candidate Files, included the following information on page one under the subheading of TEACHING:

The department should provide a separate document assessing teaching performance, with an explanation of the evaluation procedures and measures used. The department's statement should discuss both student and peer evaluations and describe the faculty member's principal areas of teaching, his or her willingness to teach courses for which there is a strong student demand, and, as appropriate, the balance between undergraduate and graduate teaching. [Italics added.]

Peer evaluation of teaching is therefore an essential part of a faculty member's promotion and tenure file according to the University of Texas at Austin. Peer observation is one part of the evaluation of teaching for improvement or for personnel decisions for merit, promotion, and/or tenure.

 

What aspects of teaching are faculty
peers most qualified to evaluate?

Cohen and McKeachie (1980) identified ten aspects of teaching that peers are most competent to evaluate about teaching. Articles on peer evaluation of teaching suggest that all of these aspects can be used during the peer review process:

A review of these aspects is facilitated by a teaching portfolio prepared by the instructor and made available to peer observers. A pamphlet entitled Preparing a Teaching Portfolio is also available at the Center for Teaching Effectiveness. In addition, peer evaluation to improve teaching or a summative peer evaluation process can include a series of classroom observations of one instructor by faculty peers and/or administrators.

 

What is peer observation?

The process of peer observation involves faculty peers that review an instructor's performance through classroom observation and examina tion of instructional materials and course design. Observations of classroom behavior are intended for reviewing the teaching process and its possible relationship to learning. The focus is on verbal and nonverbal behaviors of both the instructor and the students in the classroom.

Peer observation can produce the following evidence:

The processes of observation and evaluation require a very high degree of professional ethics and objectivity. Effective peer observation requires training in observational and analytical skills. Less subjective peer observations require time for multiple reviews.

The major strengths of peer observation are:

The major weaknesses of peer observation are:

All things considered, key authors on peer observation agree that peer observation of classroom teaching is one useful part of a peer evaluation process.

 

What are two purposes of peer observation?

Peer observations may be used for both formative feedback, for the improvement of instruction, and summative assessment for making personnel decisions. Braskamp and Ory (1994, p. 202) stated in Assessing Faculty Work that:

Peer observations are particularly useful in a program of faculty self-assessment and improvement. Instructors who wish to analyze their own teaching and student learning can benefit from a colleague's observation. Such classroom observations can be flexible and informal. In contrast, observations for personnel decision making need to be more formalized and standardized to ensure fairness, reliability, and credibility. Several trained colleagues making independent visits provide more credible summative assessment information than does one untrained colleague making a single visit.

Formative peer observation is the process of faculty members attending and observing peers in the classroom to assist with the improvement of teaching. This process may be specified by the department as a part of faculty development activities or entered into by individual faculty members who want their teaching reviewed by peers.

The formative peer observation process is most important for junior faculty as a part of the teaching improvement process in the years before tenure and promotion review. This process can prepare junior faculty early for their career teaching demands. Longitudinal studies of junior faculty stress that early teaching demands often become overwhelming and can prevent the expected devotion to quality research and service. Early intervention provides junior faculty with the tools for success in teaching as well as research and service.

Summative peer observation involves the evaluation of peer classroom behavior to provide teaching effectiveness information used for merit, promotion, and/or tenure decisions. Typically, Ad Hoc Committees on Teaching consisting of senior faculty members, junior faculty members, graduate and/or upper-level undergraduates meet individually with the instructor to be observed. The instructor and committee members review all teaching materials i.e., course materials, syllabi, exams, teaching aides, student evaluations, student advising records, and even face-to-face talks with students to obtain a complete picture of teaching before entering the classroom for an observation.

As indicated above, the committee approach to the observa tion process is somewhat different for formative and summative peer observation. Despite the differences in the processes and the objectives of the two processes, key authors believe that the development and use of an effective formative peer observation process leads naturally to a fair, objective summative peer observation process. Certainly the observation instruments developed for formative peer observation can be used for summative review in many cases. Observer skills developed in formative peer observations will be directly applicable to a summative peer evaluation process.

Although the outcomes of formative and summative observation are different, effective formative peer observation serves as a vehicle for effective summative peer observation and evaluation at the departmental level. Most authors believe that the two processes are compatible and mutually supportive of faculty involvement in either formative or summative peer observation.

 

How do you choose or design
a peer observation instrument?

Following guidelines established by the department or school will ease many feelings of misgiving about the peer observation process. An organizational plan that includes both a set of departmental observation forms and a departmental or division Ad Hoc Committee on Teaching, helps validate the peer observation process with all faculty. The three most common ways to document or guide formative or summative peer observations are: checklists, rating scales, and written analyses.

Types of Instruments:

Peer evaluations of a faculty member's classroom behavior can be based on checklists, rating scales, and/or written analyses. Each type of document has it's strong and weak points as a format for peer observa tion.

Checklists focus the observer's attention during the observation and also, the instructor reviews the checklist before the classroom visit to understand observation expectations. Checklists are often viewed as too restrictive by both observers and those observed. Checklists also only indicate that a behavior has been observed with no feedback about the effectiveness or lack of effectiveness of a specific behavior. Checklists are similar to rating scales with no scoring involved. A checklist can be created from a rating scale form by deleting the scale and asking the observers to record observed behav iors. See Appendices.

Rating scales also focus the attention of the observer and the instructor but add the dimension of indicating relative effectiveness of a given behavior. All rating sheets need to include behavior descriptors and an explanation of the scale points. All instruments must be clearly relevant to the classroom teaching situation. Checklists and rating sheets include specific language for instructors to react to or take action on . See Appendices.

The written analysis format affords an open-ended opportunity for a peer not only to select what to observe, but also, how to interpret the information and structure the evaluation. The down side to written appraisals is that peers can comment on very limited behaviors, focus only on one criteria, or reflect only the observer's personal approach to teaching. See Report of Classroom Observation in the Appendices. (This online document requires an Adobe Acrobat PDF reader.)

It is suggested that a combination of checklist, rating sheet, and written analysis formats be adopted and used for formative and summative observations. All observation forms should yield accurate insights into the classroom environment and obtain data typical of classroom behavior. See Appendices.

One checklist, rating sheet, or written analysis form is not likely to cover all teaching demands in any one department. For example, one form is not likely to address teaching a large lecture class, teaching a special projects class, and teaching a laboratory class. Some of the same variables may appear in all three teaching situations, but each environ ment has specific teaching demands that the others do not.

 

Formative Peer Observation Process

Some basic types of formative peer observation are: a master faculty program, mentor-mentee pairs, peer development triads, graduate student feedback, small group instructional diagnosis, and the appraisal interview.

Master Faculty Program

Katz and Henry (1988) promoted a Master Faculty Program that paired a successful senior professor with a junior faculty member to collaborate on teaching; to observe each other's classes to learn; to compare and improve teaching methodologies; and to foster weekly discussions about effective teaching. These "buddy system" collaborations provide many rewards for both faculty members involved. With information gathered in the observations, faculty pairs meet once a week or so to discuss how student learning has been fostered or hindered in the learning methodolo gies and to share insights about improving teaching.

Mentor-Mentee Relationships

Roles played by mentors include friendship for emotional and personal support, career guidance for increased professional visibility, information source for discussing departmental and university expectations, and/or intellectual guidance to provide research and writing reviews. More departments are arranging mentor-mentee pairing thereby giving new faculty the greatest opportunity to prove their worth and fulfill institutional expectations. Mentors are generally selected from the same discipline but Boice suggests that effective mentoring does not have to be discipline bound. Boice (1992) found that:

Master Faculty and mentor-mentee programs are very similar in organization, but mentors are not necessarily designated as "master teachers" and may be chosen directly by the mentees for reasons other than teaching expertise.

Peer Development Triads

Peer development triads extend the "pair concept" and offer addi tional opportunities to share and compare teaching/learning strategies with two peers.

Graduate Student Feedback

An example of a graduate student feedback mechanism can be found in The University of Chicago's Graduate School of Business. They designed a one hour MBA course for graduate students in which they provide feedback to instructors by auditing a professor's classes, videotaping selected presentations, and gathering suggestions from enrolled students for midsemester course changes.

Small Group Instructional Diagnosis

The Small Group Instructional Diagnosis (SGID) is another method used to improve instruction with the aid of a peer or faculty development consultant. The process, which can be easily learned by peers, is de scribed by Bennett (1987) as follows:

Appraisal Interview

The appraisal interview is used by chairs who want to discuss a teaching problem with an instructor. First the chair needs to create a supportive environment for the interview and begins with questions about how things are going in general. The chair may share some insights from her/his classroom observations to offer encouraging comments about the instructor's practices. Then the chair asks whether the instructor is having any difficulties. If the instructor does not mention difficulties, the chair then can refer to information taken from their own classroom observations or problems raised by students. Finally, the chair asks how she/he or the department can help the instructor solve the problem. The appraisal interview must be handled carefully and more than one meeting may be required to bring about the necessary modifications. The primary objective of any type of formative observation process is improvement of teaching.

 

Summative Peer Observation Process

The three person committee, faculty, student, and/or administrator, or the Ad Hoc Committee on Teaching, is the most frequently used arrangement for summative peer observation. A larger committee becomes too cumbersome and a smaller committee does not provide enough data. The Ad Hoc Committee can be composed of nominations made by the instructor and the chair/dean; this selection process is particularly helpful for promotion and tenure decisions. Shared nomina tions provide the instructor the opportunity to recommend one or more observers for the committee. Preferably, the committee members will remain anonymous to each other and the general departmental faculty, in order to avoid contamination of observations. A summary of the three faculty/administrator/ student observations should be provided by the committee chair.

Summary reports based on checklists, rating forms, and/or written analyses should include the following information (Centra, 1993, p. 130):

  1. Classroom performance observation forms

  2. Instructional materials review

  3. Advising activity review

  4. Participation on graduate committees and graduate teaching

  5. Special recognition for teaching

  6. Overall recommendation

Protocol for Summative Peer Observation Committee Members

It is suggested that each Ad Hoc Committee member follow this protocol for summative peer observations (Braskamp & Ory, 1994; Millis, 1987).

 

What are some key issues to remember
about formative or summative peer observation?

There are key issues to remember as you enter a formative or summative peer observation process:

It is recommended that peer classroom observations be used as simply one part of the larger picture with regard to evaluation of teaching effectiveness. Do not give peer observations undue weight in summative evaluations for the following reasons:

 

What are some recommendations for
institutional use of peer observations?

Braskamp and Ory (1994, pp. 205-206) list the following suggestions for adopting peer observation:

 

If your department wants to develop
peer observationas a part of peer evaluation,
what should you do?


Suggestions for developing a peer observation process as part of the peer evaluation process in your department are:

 

References:

Bennett, W.E. (1987) "Small Group Instructional Diagnosis: A Dialogic Approach to Instructional Improvement for Tenured Faculty." The Journal of Staff, Program, and Organization Development, 5(3), 100-104.

Boice, Robert (1992) The New Faculty Member. San Francisco: Jossey-Bass Publishers.

Braskamp, Larry A., Brandenburg, Dale C., and Ory, John C. (1984) Evaluating Teaching Effectiveness: A Practical Guide. Beverly Hills; Sage Publications, Inc.

Braskamp, Larry A. and Ory, John C. (1994) Assessing Faculty Work: Enhancing Individual and Institutional Performance. San Francisco: Jossey-Bass Publishers.

Cohen, P.S. and McKeachie, W.J. (1980) "The Role of Colleagues in the Evaluation of College Teaching." Improving College and University Teaching, 28, 147-154.

Centra, John A. (1993) Reflective Faculty Evaluation. San Francisco: Jossey-Bass Publishers.

Katz, J. and Henry, M. (1988) Turning Professors into Teachers: A New Approach to Faculty Development and Student Learning. New York: Macmillan.

Millis, Barabara J. (1987) "Colleagues Helping Colleagues: A Peer Observation Program Model." The Journal of Staff, Program, and Organization Development, 7(1), 15-21.

Seldin, Peter (1984) Changing Practices in Faculty Evaluation: A Critical Assessment and Recommendations for Improvement. San Francisco: Jossey-Bass Publishers.

See the Bibliography for more extensive readings on peer observation and evaluation.

 

Bibliography for Peer Observation & Evaluation

 

Arreola, R. A. “Defining and Evaluating the Elements of Teaching.” In W. Cashin (Ed.) Proceedings of the Sixth Annual Academic Chairperson’s Conference, Center for Faculty Evaluation and Development, Kansas State University, Spring, 1989, 32, 3-12.

Arreola, R. A. Developing a Comprehensive Faculty Evaluation System. Bolton, Mass.: Anker Publishing Co., 1995.

Baldridge, J. V., Kemerer, F., and Associates. Assessing the Impact of Faculty Collective Bargaining. ASHE-ERIC Higher Education Report, no. 8. Washington, D.C.: Association for the Study of Higher Education, 1982.

Blackburn, R. T., and Clark, M. J. “An Assessment of Faculty Performance: Some Correlates Between Administrators, Colleagues, Student, and Self-Ratings.” Sociology of Education, 1975, 48, 242-256.

Braskamp, L. A. and Ory, J.C. Assessing Faculty Work. San Francisco: Jossey-Bass, 1994.

Braxton, J. M., Bayer, A. E., and Finkelstein, M. J. “Teaching Performance Norms in Academia.” Research in Higher Education, 1992, 33(5), 533-569.

Carnegie Foundation for the Advancement of Teaching. National Survey of Faculty, 1989. In Boyer, E. L. Scholarship Reconsidered: Priorities of the Professoriate. Lawrenceville, N. J.: Princeton University Press, 1990.

Centra, J. A. “Colleagues as Raters of Classroom Instruction.” Journal of Higher Education, 1975, 46, 327-337.

Centra, J. A. How Universities Evaluate Faculty Performance: A Survey of Department Heads. GREB Research Report, no. 75-56R. Princeton, N. J.: Educational Testing Service, 1977.

Centra, J. A. Determining Faculty Effectiveness. San Francisco: Jossey-Bass, 1979.

Centra, J. A., Froh, R. C., Gray, P. J., and Lambert, L. M. A Guide to Evaluating Teaching for Promotion and Tenure. Acton, Mass.: Copley Publishing Group, 1987.

Chickering, A. W., and Gamson, Z. (Eds). Applying the Seven Principles for Good Practice in Undergraduate Education. New Directions for Teaching and Learning, no. 47., San Francisco: Jossey-Bass, 1991.

Clift, J. and others. “Establishing the Validity of a Set of Summative Teaching Performance Scales.” Assessment and Evaluation in Higher Education.. 1989, 14(3), 193-206.

Cohen, P. A., and McKeachie, W. J. “The Role of Colleagues in the Evaluation of College Teaching.” Improving College and University Teaching, 1980, 28, 147-154.

Cross, K.P. “Classroom Research: Helping Professors Learn More About Teaching and Learning.” In P. Seldin and Associates (Eds.), How Administrators Can Improve Teaching: Moving from Talk to Action in Higher Education. San Francisco: Jossey-Bass, 1990.

Diamond, R. M. Serving on Promotion and Tenure Committees; A Faculty Guide. Bolton, Mass.; Anker Publishing Co., 1994.

Diamond, R. M. Preparing for Promotion and Tenure Review; A Faculty Guide. Bolton, Mass.: Anker Publishing Co., 1995.

Dilts, D. A., Haber, L. J., and Bialik, D. Assessing What Professors Do. Westport, CT.: Greenwood Press, 1994.

Dressel, P. L. Handbook of Academic Evaluation: Assessing Institutional Effectiveness, Student Progress, and Professional Performance for Decision Making in Higher Education. San Francisco: Jossey-Bass, 1976.

Earl, S. E. “Staff and Peer Assessment–Measuring an Individual’s Contribution to Group Performance.” Assessment and Evaluation in Higher Education, 1986, 11 (1), 60-69.

Feldman, K. A., “Instructional Effectiveness of College Teachers as Judged by Teachers Themselves, Current and Former Students, Colleagues, Administrators and External (Neutral) Observers.” Research in Higher Education, 1989, 30, 137-189.

Flygare, T. J. “Board of Trustees of Keene State College v. Sweeney: Implications for the Future of Peer Review in Faculty Personnel Decisions.” Journal of College and University Law, 1980, 7, 100-110.

French-Lazovik, G. “Peer Review: Documentary Evidence in the Evaluation of Teaching.” In J. Millman (Ed.), Handbook of Teacher Evaluation. Newbury Park, Calif.: Sage, 1981, 73-89.

Higher Education Reserach Institute. The American College Teacher: National Norms for the 1989-90 H.E.R.I. Faculty Survey. Los Angeles: University of California, 1991.

Kremer, J. “Constant Validity of Multiple Measures in Teaching, Research, and Service and Reliability of Peer Ratings.” Journal of Educational Psychology, 1990, 82, 213-218.

Love, K. G. “Comparison of Peer Assessment Methods: Reliability, Validity, Friendship, Bias, and User Reaction.” Journal of Applied Psychology, 1981, 66, 451-457.

McGaughey, R. A. “Why Research and Teaching Can Coexist.” The Chronicle of Higher Education, Aug. 5, 1992, 48 (38), A36.

Murray, H. G. “The impact of Formative and Summative Evaluation of Teaching in North American Universities.” Assessment and Evaluation in Higher Education, 1984, 9, 117-132.

Ory, J.C., and Parker, S.A. “Assessment Activities at Large, Research Universities.” Research in Higher Education, 1989, 30 (4), 375-385.

Root, L. S. “Faculty Evaluation: Reliability of Peer Assessments of Research, Teaching, and Service.” Research in Higher Education, 1987, 26, 71-84.

Seldin, P. and Associates. Improving College Teaching. Bolton, Mass.: Anker Publishing Co., 1995.

Tobias, S. “Peer Perspectives on the Teaching of Science.” Change, March/April 1986, 36-41.

Webster, D. S. “Instructional Effectiveness Using Scholarly Peer Assessments as Major Criteria.” Review of Higher Education, 1985, 9 (1), 67-82.

Weimer, M. Improving College Teaching. San Francisco: Jossey-Bass, 1990.