home
Assess teaching

Analyzing observational data

Use these analysis methods for formative evaluation:

Example: Checklist

To better organize her lectures, Professor Richards developed a checklist, which she revised after discussion with a colleague who agreed to observe her class on three occasions. These were her colleague's observations after the first occasion:

The instructor...

 

 

Comments

1. stated the purpose of the lecture.

_X_ Yes

___ No

stated clearly at start of class

2. explained the relation of the class to the previous one.

_X_ Yes

___ No

very clear and concise

3. put class objectives on a PowerPoint slide

_X_ Yes

___ No

good, reinforced #1

4. verbally provided an outline of lecture content.

_X_ Yes

___ No

with PowerPoint slide

5. made transition statements between lecture segments

___ Yes

_X_ No

mostly jumped to new topic

6. summarized periodically and at the end of class.

___ Yes

_X_ No

some at end, but otherwise no

7. connected different points or topics in summaries.

___ Yes

_X_ No

would be helpful to tie together

The colleague explained the ratings and written comments, and he and Professor Richards discussed their impressions of the class's organization. The colleague completed the checklist during two more observations and discussed his impressions with Professor Richards after each. After completing all three observations, he wrote a summary, noting improvements and areas for continued work.

Example: Ratings

To increase student involvement during lectures, Dr. Smith decided to ask more questions, leave more time for discussion, and include interactive exercises. He explained his goals and gave a copy of the syllabus and a lecture outline to a colleague, who completed ratings after each observation:

1. The instructor's questioning skills

1-poor 2-fair 3-average 4-very good 5-excellent
Comment: Dr. Smith asked thought-provoking questions and waited for students to consider them before responding.

2. The instructor's skill in responding to questions

1-poor 2-fair 3-average 4-very good 5-excellent
Comment: He made sure students understood his answers and the whole class heard questions and understood their relevance.

3. How engaged students are in interactive exercises

1-not at all 2-fairly 3-very 4-completely
Comment: It varied somewhat from student to student, but many students seemed bored or unsure about what to do.

4. The instructor's skill in facilitating group work

1-poor 2-fair 3-average 4-very good 5-excellent
Comment: He did a good job at asking and answering questions but did not adequately address confusion or signs that students were not engaged.

Dr. Smith totaled the ratings (4 + 4 + 2 + 3 = 13) for the observation to compare it to later observations. After the second observation, Dr. Smith met with his colleague to discuss the ratings and their impressions of strengths and weaknesses. The colleague observed two more class sessions after Dr. Smith made changes and discussed the results with Dr. Smith.

Example: Narrative

Dr. Paderas, who taught morning and evening sections of the same course, wondered why she had fallen far behind the syllabus in her evening section. She asked a colleague to observe both sections and complete narrative logs. Like videotaping, a narrative log is a record of verbal and nonverbal behaviors but is less intrusive and provides another person's perspective. A portion of the logs for the first observation pair follows:

Morning lecture

8:50:

Dr. Paderas arrives ten minutes before the start of class, sets up a PowerPoint presentation, and organizes her lecture notes. Two students talk with her about the exam next Thursday, and she explains that she will hold a review session early next week.

9:00:

Class begins promptly with all but five students present. Dr. Paderas reviews the last lecture and presents the objectives for this class: understanding the possible causes and treatments for schizophrenia. She asks if there are questions about the previous lecture, and one student asks about the difference between dissociative identity disorder and schizophrenia. Dr. Paderas briefly describes dissociative identity disorder and explains key differences from schizophrenia, saying that she will provide more detail in today's lecture.


Evening lecture

4:59:

Dr. Paderas arrives one minute before the start of class. She has some difficulty organizing lecture notes because she is carrying more papers and books than during the morning lecture. Two students approach her to discuss their grades from the last exam. Dr. Paderas talks briefly with them but explains that she has to review her grade book to be certain about their grades. Two other students explain that they come directly from work to class and that recent job demands kept them from attending the last lecture. One of the students asks if this will affect his participation grade and Dr. Paderas explains that it will not as long as he does not miss more than four lectures.

5:06:

Class begins six minutes late with 10-15 students absent. Dr. Paderas reviews the last lecture and presents objectives for today's lecture about schizophrenia. She asks if there are questions about the previous lecture, and one student says she was confused about dissociative fugue. Dr. Paderas explains the concept and follows up to make sure the student is no longer confused. During the explanation, three students arrive and, as they sit, exchange comments with friends, which makes it difficult to hear Dr. Paderas. One student who arrives late asks Dr. Paderas to review the objectives for the lecture.

Dr. Paderas and her colleague discussed time management issues after the first set of observations. Dr. Paderas asked the colleague to observe a lecture in both sections two weeks later to evaluate the extent of improvements.

Additional Information:

 Center for Teaching Excellence. (undated) Preparing for Peer Observation in, TX: The University of Texas at Austin.

Chism, N.V.N. (1999). Peer Review of Teng: A Sourcebook. Bolton, MA: Anker Publishing Co., Inc.

Page last updated: Sep 21 2011
Copyright © 2007, The University of Texas at Austin