MARCH 17, 2014


A. Report from the Educational Policy Committee on Required and Proposed Revisions to the Basic Form Course Evaluation (D 11450-11460).
Professor Mary Rose (committee chair, sociology) prefaced her PowerPoint presentation (reference Appendix A) saying the fall semester had been “consumed” by the business of modifying the course evaluation form and added that “in quite a wonderful way faculty and students have very strong opinions about what our course evaluation form should look like. So I won’t be surprised if people here also have opinions.” She then explained that UT System had mandated five particular items on the evaluation forms, which she said thankfully were the same as what is currently being used by UT Austin, but that they had to be worded in a specific way: 1) the instructor clearly defined and explained the course objectives and expectations; 2) the instructor was prepared for each instructional activity; 3) the instructor communicated information effectively; 4) the instructor encouraged me to take an active role in my own learning; 5) the instructor was available to students either electronically or in person.  Professor Rose said the committee had decided to take this opportunity to update the forms, particularly the basic form used by the majority of faculty members. The committee had been asked to address specific issues, for example, students requested one item be added that would assess whether the assigned supplemental materials (i.e. text books) were useful to the course. Their rationale was the high cost of some textbooks. Another request had come from the fine arts faculty who felt penalized by some questions, which were not applicable to their school (e.g. having tests and assignments returned quickly). Students often responded to these kinds of questions as “neutral” resulting in an artificially low score. Professor Rose explained that the current form has two sections, the first section having six items, some of which would have to be deleted to accommodate the five mandated items; the second section having five items. She said the Center for Teaching and Learning (CTL) had agreed to allow eight questions in the first section. Besides the five mandated questions, the three other questions would be 1) the instructor made me feel free to ask questions, disagree and express my ideas; 2) the instructor was genuinely interested in student learning; 3) the course materials (e.g., text and supplemental materials) were helpful to me. The committee had decided to delete four questions from section one 1) the course was well organized; 2) the instructor showed interest in the progress of students; 3) the tests/assignments were usually graded and returned promptly; 4) At this point, I feel that this course will be (or has already been) of value to me. The rationales for each deletion is provided on slide five in Appendix A.

Professor Rose described section two of the basic form as having two overall ratings: 1) the overall rating of the instructor, and 2) the overall rating of the course. In addition, she said it asked for the student’s GPA and for their expected grade in the class. Feedback indicated that the two overall items should be kept. In regard to the “workload” question, Professor Rose said the committee felt that it was actually measuring two things, “Is it like your other courses and that’s the average thing; is it average in the sense that it’s familiar to you versus a sort of normative assessment of too high or too low?” She said the committee had decided to take a more normative approach and “tighten up” the response categories to have more variability and proposed changing the categories to excessive, somewhat high, about right, somewhat light, and insufficient. Regarding the expected grade question, the committee felt the words “to date” were redundant and proposed deleting them so the question would read “My probable grade in this course is.” Chair Hart voiced concern about the language and suggested feedback from experts as students with English as their second language might not understand its meaning. She said it seemed to her to be “quite colloquial.” Professor Rose responded saying that “it was a very common way of asking the question, a traditional measurement.” Professor Olson suggested saying sufficient instead of right.

Professor Sonia Seeman (music) asked if a category “Applicable” could be added as some of the questions might not apply. Professor Rose replied that in the first section, the first five items could not be touched since they had been mandated by UT System; therefore, to provide a new response category, a whole new section would have to be created for the remaining three questions. What Dr. Rose suggested instead would be to clearly state in the instructions that students skip questions that do not apply. She emphasized that an “issue in measurement is always that when you try to fix one problem, you’re going to create others. So you’re likely to have people skip things when they’re not supposed to.” Professor Rose added that the committee had recognized the concern about this issue and felt it could be handled in the instructions to students. Professor Seeman disagreed by saying it was problematic and should be reworked so that “one size fits all” and added that if a student skipped a question, whether because it didn’t apply or because they just skipped it, data would not be gathered. Professor Rose concurred and said that if the University had more autonomy, it would be something the committee could entertain.

Professor Novak expressed concern about the ordering of the categories, going from bad to good on one slide and good to bad on the others. He felt that it would cause “endless confusion and wrong answers.” Professor Rose acknowledged that the Faculty Council Executive Committee had brought up the same concern. However, she explained that if the “overall” items are left untouched, and the mandated items from System cannot be modified, then the only way around it would be to start off the “probably grade” with F, which she felt was “out of balance.” She added that the proposal was “entirely consistent with how we’ve been doing it for years.” Professor Novak argued that the ordering should be consistent on every question throughout the answer sheet. “So if System dictates the top order, than we ought to reorder our answers to fit that order throughout.” Professor Rose said her understanding was the questions from System could not be “touched” in any way, to which Professor Novak responded, “Well this doesn’t touch them, it just reorders them.” Later, Professor Daniel Sutherland (art and art history) also voiced his agreement with Dr. Novak. Dr. Hart asked for clarification whether the only question out of order was the one relating to the letter grade. Professor Rose confirmed that was correct.

Professor Kurt Weyland (government) said he did not like the question “The instructor was generally interested in student learning.” He felt that it would be much more important to assess what the student learned with a question like “I’ve learned a great deal in this course.” He also conveyed that he liked the organization question. He said that when assessing a colleague’s teaching evaluation, this question could be helpful as one “can spot bad teachers—those who are not organized. Because people can make all kinds of excuses about students not liking them or what not, but [if] they have a low rating on organization, you know that they are not putting an effort into it.” Professor Novak concurred saying that the question “is important to students, it is important to department chairs.” He added that he did not think having two extra items on the form would diminish the response rate. Professor Rose replied that, in this case, it was about space and asked what question would he eliminate to get the organization question back on the form? Professor Weyland said his biggest priority would be to assess how much the students had learned and said there was not a question like that on the form. He said, “I think that’s really crucial.” Having agreed with Professor Weyland, Professor Novak suggested that question #2 was “kind of a meaningless question” as it “requires the student to psychologically project what the instructor thinks.” Professor Rose pointed out that Professor Weyland had suggested rewriting that question “so students would say that they felt that they learned.”  Professor Novak suggested omitting the question “the course is of value to me, or I learned a lot, or one similar, ” to which Professor Rose responded saying the committee had proposed that it be eliminated. She stressed that the basic form was in the hands of the Faculty Council, and it would be for the Council to decide what questions should be on the form. But, she urged members to be cognizant of the space limitation. To allow more questions, Professor Reichl suggested questions be edited so they would be two lines long. Professor Rose said she would investigate with CTL to see if that would be allowed. Professor Weyland suggested the space for comments could be placed on the back of the form to allow more room. Professor Rose said the committee had considered reducing the comments space but felt it would discourage comments. She again reiterated that it was the Council’s decision to make. Professor Donna Kornhaber (English) voiced her support of a question that would ask students to evaluate how much they had learned. She added that students often wrote those thoughts in the comments section, but there was no place for that to be reflected on the form. She also agreed that question #2 ought to be eliminated since it seemed to be “very subjective, very personality based.” She suggested replacing it with something students would write in the comments section.

Chair Hart reminded those present that the committee would take the Council’s feedback into account and a that revised basic form would probably be submitted for a vote in April.  Professor Beckner brought to the Council’s attention that UT System was only interested in receiving data on the five mandated questions. Mary Rose added that the memo had also encouraged electronic course evaluations, which the committee didn’t not address this year. Professor Rose closed by saying the committee would investigate the possibility of adding one more item on course recognition to make it a total of nine items in the first section.

Return to main minutes