Teaching Assessment: Practices for Graduate Student Instructors
The Individual Consultation Process at DIIA/Teaching
Effectiveness
by Karron G. Lewis, Ph.D.
One-to-one consultation is probably the most effective method available to instill long-term changes in the teaching techniques of individual faculty members. The teaching excellence staff members have been trained in objective observation techniques and are skilled in the analysis of teacher-student interactions which take place during a class session. They are also skilled in the analysis of course materials and typically provide feedback to each instructor concerning the effectiveness of the handouts, exams, etc. used in the course.
When first contacted by a faculty member a pre-observation conference is scheduled. The faculty member is asked to bring texts, their current syllabus, past exams, etc. to help the consultant get an idea of the content and cognitive level of the course. Copies of past student evaluations are also requested to provide some idea of how the students have felt about the course and instructor in the past.
During the pre-observation conference, the consultant and faculty member discuss the concerns/questions which prompt the faculty member seek assistance. This is also a time for the consultant to get to know the faculty member and vice versa. At the end of this conference, the consultant and faculty member will decide on a time for the in-class observations to occur.
Normally three (3) in-class observations are made before the consultant discusses these observations with the faculty member. (Unless the consultant sees something which can be changed easily or wishes to comment on techniques which were especially effective.) By making at least three observations, the consultant can feel certain that what is happening in the class is representative of the normal teaching process. If only one observation was made, it might fall on a "bad" day and thus not be a true picture of the faculty member's teaching techniques.
During the first observation the consultant typically takes notes as if he/she were a student to determine the pace of the lecture, the organization and clarity of the material presented, and so forth. A general observation form is also filled out during this class session (see Example 1) to provide information about the faculty member's teaching style, the type of classroom, and the attentiveness and characteristics of the students.
During the second and third observations the consultant may code the verbal interactions which take place using the Cognitive Interaction Analysis System (CIAS). The categories for this system are shown in Example 2. One category is recorded every three seconds or whenever the interaction changes, whichever occurs first (see sample Coding Sheet in Example 3). These codes are then entered into a computer and a matrix of the interactions is generated (Example 4). The consultant will then analyze the coding forms and the matrix to determine the types of interaction patterns typically used.
Utilizing all of the written course materials and in-class observation information, the consultant then writes out a detailed description of what was observed along with an analysis of the probable sources for the concerns expressed in the pre-observation conference. Then another conference session is scheduled in which all of the above information will be shared with the faculty member. Together the consultant and faculty member will decide how to facilitate the changes the faculty member wishes to make in his/her teaching and training sessions for the development of more complex teaching skills may be scheduled. If the faculty member feels the changes can be made without these training sessions, then methods for accomplishing these changes are discussed and a follow-up observation time is scheduled (typically 3-4 weeks later).
During the follow-up observation the consultant will focus on the skills and techniques identified for possible change or improvement. Usually the verbal interactions are coded again, analyzed, and discussed with the faculty member. Another detailed memorandum is written and a decision is made about whether the process should continue.
The sequence of events outlined above can occur as many times as needed until the faculty member is satisfied with his/her teaching. (Sometimes only one or two sessions are required and sometimes an extended period of assistance is requested -- e.g., several semesters.)
Because the primary tool used in this process is an objective observation system, it provides some convincing "hard data" to illustrate some possible causes for particular problems. The first reaction of most faculty members to this tool is amazement. Then, after the significance of the data are explained to them, they become excited about the information it can provide. The most significant change which is usually seen immediately is a heightened awareness of the students and their role in the learning process. Interaction is now encouraged and often the faculty member expresses a desire to learn how to question at higher cognitive levels.
In the time during which this consultation process has been used there have only been one or two faculty members who reacted negatively to the "hard data." (CIAS can also be used with videotaped class segments. The videotape provides conclusive evidence that "there was no silence after that question so the students didn't have an opportunity to think of an answer.")
Videotape may also be used in the training sessions for complex teaching skills, mentioned on page 1. In these sessions, the faculty member teaches a short lesson, 5-10 minutes in length, during which the teaching skill to be improved is practiced. The faculty member and consultant then view the videotape, analyzing the progress made in learning the desired teaching skill. As with the in-class observations, this process may be repeated as often as necessary until both the faculty member and consultant are satisfied that the desired skill has been learned.
As in any one-to-one consultation process, the needs of the client determine the methods used. Thus, each consultation process may be a little different. In most cases our consultants are able to assist the faculty members who request our services and, typically, those with low student ratings are able to document changes in the data that is collected.