home
Assess technology

Example: Assessment plan

Course lecture webcast project fall 2005

Project description

With the initiative of Technology Enhanced Learning, the course lecture webcast projects began with three large lecture-based geology courses as test cases in fall 2004 and continued with two lecture-based government courses in spring 2005, In fall 2005, the same three geology courses are webcast.

Currently, webcasts are not intended to replace in-class lectures but to serve as an additional source through which students can view lectures repeatedly or when they miss classes. Therefore, no existing instructional activities—such as in-class lectures, labs, office hours, or in-class quizzes and exams—are eliminated.

The webcasts are delivered as streaming videos via Blackboard, through which students can access materials for only the courses in which they are enrolled. Students can view lecture videos integrated with visual and audio materials at any time (live or archived) and as many times as they want on the Internet during the semester. Students have several viewing options: they can change the pace at which they watch using stop, double speed, or slow speed, and they can go directly to the portion of the video they wish to view using a slide list or playhead. They can also enlarge the lecture video window or slide window.

Stakeholders

Central questions

  1. How does webcasting affect students’ class attendance?
  2. How does webcasting affect students’ performance?
  3. How does webcasting affect students’ learning experiences?
  4. Why do students use webcasts?
  5. How do students use webcasts?
  6. What types of students benefit from webcasts?
  7. How does webcasting affect instructors?
  8. How do instructors transform their courses when their courses are webcast?
  9. What types of expectations does webcasting set up for students and instructors?
  10. What are the adverse effects of webcasts for students and instructors?
  11. What are the technical issues/problems for students and instructors?
  12. What are the most important features of the webcast interface?
  13. Can webcasting recreate the large lecture-based classroom experiences?
  14. What are the implications for administration and IT support?

Assessment purpose and uses

The purpose of assessing the webcast program is to explore holistically the merit and challenges of webcasting large lecture courses, including the perspectives of students, instructors, and TEL program staff. The results will be used to help decisions on whether or not the webcast service should be expanded and in what form webcast services should be delivered in the future.

Project components

Table 1 shows the instruments and methods used for data collection to address central questions using multiple data sources to be triangulated to yield less biased conclusions.

Project Responsibilities

Evaluators are responsible for the overall management of the project as well as instrument creation and administration, data management and analysis, and the reporting of findings. Program staff is responsible for providing evaluators timely, periodic feedback to evaluators. Timely feedback from program staff is critical to the development of valid evaluation instruments and in meeting target dates. In addition, program staff will assist evaluators in attendance counts, conducting in-class surveys, and providing audit data and any information based on observation in the course of webcast operation. Program staff will have an opportunity to review and comment on all evaluation instruments and drafts of written reports.

Project Timeline

Table 2 shows a time line for data collection activities.

Table 2 – Webcast Fall 2005 Assessment Timeline

Begin date End Date Task Notes

8/25/05

12/20/06

Instructor interviews

At the beginning and end of the semester

8/25/05

12/23/05

Support observations

Throughout the semester as necessary

8/25/05

12/23/05

Audit data

Throughout the semester

8/31/05
(the first day of
class)

12/20/05
(the last final exam
day)

Access data

Throughout the semester

9/19/05
(after 12th day)

12/9/05
(the last class)

Attendance counts

On most lecture class days

9/19/05

12/9/05

Course observations

Throughout the semester as necessary

11/4/05

11/11/05

Mid-semester survey

Online survey

11/10/05

11/18/05

Usability testing

Usability tests and student interviews are conducted together.

11/10/05

11/18/05

Student interviews

11/28/05

1/16/06

End-of-semester survey

In-class paper survey and online survey

12/15/05

12/23/05

Collecting performance data

After the final exam

Page last updated: Sep 21 2011
Copyright © 2007, The University of Texas at Austin