Understanding the Course Instructor Survey (CIS) Results

 

Karron G. Lewis, Ph.D., CTE

Steve Fitzpatrick, Ph.D., MEC

 

It is hard to imagine anyone on the teaching staff at UT Austin who has not at some time gazed with a puzzled frown at the results of a course-instructor survey from some class.  Those of us at the Center for Teaching and Learning Excellence (CTLE) and Instructional Assessment and Evaluation (IAE), formerly known as the Measurement and Evaluation Center (MEC), are familiar with this puzzlement and we empathize with faculty members’ dismay and confusion about what to do with the CIS results. 

 

This document is being provided with your CIS results to help you interpret them and decide how you might use the student ratings and comments to make your teaching more effective.

 

Interpreting Numerical Feedback

 

When you get your CIS feedback forms, before you do anything else, sort them into stacks based on the rating each student gave for the “Overall Course” or “Overall Instructor” ratings.  Then, look at the stacks in order from highest overall course rating to lowest rating.  This will help you put the student ratings and comments in perspective.  For example, it will help you see if students who rate the course high also say you are somewhat unorganized, or if only those who rate the course low say that.

 

The results summary sheets you receive from IAE include the following information for each item:

 

Frequency distribution – This shows the number of students who responded to each option.  As you look at these, note how many responses are in the  “agree” and “strongly agree” categories, and how many are in the  “disagree” and “strongly disagree” categories.   Note the particular items that have responses in the “disagree” and “strongly disagree” areas.  These items indicate things you may want to consider changing in some way.

 

Mean – The number in the last column on the right is a mean based on the 5-point scale (1=Strongly Disagree and 5=Strongly Agree).  You should look carefully at items that have means of less than 3.5 and consider changes in those areas. For information about what changes you may want to consider, see the section on “The Research Behind Student Evaluations” on the CTLE website at http://www.utexas.edu/academic/cte/feldman/index.html. 

 

To compare your means to means for the overall university or for your college, go to http://www.utexas.edu/academic/mec/cis/index.html and click on “CIS College Summaries.”

 

To chart your progress over time, you can set up an Excel spreadsheet and graph the means for each item by year.  This will help you identify changes over time (either up or down) and you’ll be able to alter your teaching in a timely manner.  The CTLE Staff can help with this.

 

 

Class Size vs. Number of Responses – At the top of the results sheet the enrollment in the class is given.  In the next to the last column, the number of students who filled out the CIS form is given.  The survey results may not be valid if the percent of students who responded to the survey is less than shown below:

                               

Class Size                  Minimum Response Rate

5-20                                                                                           80%

21-30                                                                                        75%

30-50                                                                                        66%

50-100                                                                                     50%

More than 100                        50%

 

For example: If you had 16 students in your class and 14 completed the survey,  the response rate is 87.5%. This suggests that the data are valid.  On the other hand, if you had 120 students in your class and only 40 (33.3%) filled out the CIS forms, your data are probably not valid and you should interpret the results with care.

Things to keep in mind: Students tend to rate classes of …

15 students or less, highest

16 to 35 students, next highest

More than 100 students, next highest

35 to 100 students, least favorable

Major courses and electives receive higher ratings.

Required courses receive lower ratings.

Humanities courses are typically rated higher than physical sciences courses.

 

Interpreting Written Feedback

When looking at student written feedback, it is again most informative to sort the forms according to the students’ ratings on either the “Overall Course” or “Overall Instructor” item.  Read the comments from those students who rated you or the course the highest first and then move to the others.  This will give you a more realistic view of how many students thought that you and/or the course are excellent and how many weren’t too happy with how things went in the course.  Often if you don’t sort the forms first, you tend to just look for the negative comments and, even though there may only be 4 or 5 out of 30, you may say, “My class hated me this semester!”

 

In addition, you can categorize the student comments as demonstrated in the grid below.  This grid will help you focus on whether the student comments relate to the subject matter, organization/clarity, interaction, or dynamism/enthusiasm.  (Items in italics are positive comments; those not in italics are negative comments; numbers in parentheses indicate number of students who made similar comments.)  In the example below, note how often and by which student rating level mention was made of lack of organization.  (Hint: 4, 2, and 1)  Perhaps this instructor should consider how to become more organized, because even students who rated the course above average said more organization was needed.