last updated: Nov 18 2008
Web Site Questions and Comments
The University of Texas at Austin
Executive Vice President and Provost

Outcomes Assessment FAQs

Concepts & Terminology

What is the relationship between formative and summative assessment?

Formative assessment means to form or shape the program or performance while summative means to make judgments about the result. Formative assessment is generally used in an ongoing fashion to make changes and improvements during the course of a program. Summative assessment is used to make conclusions about a program’s value and whether it should continue.

What is the difference between a program goal and a program outcome?

Basically, program goals are broad general statements of what you want to accomplish at the program level. Program outcomes specify the measurable knowledge, skill, or behavior that you expect program participants to be able to demonstrate.

How are program outcomes different from course objectives?

Course objectives refer to what content individual instructors intend students to know or be able to demonstrate by the end of a specific course. Course objective are usually very specific and limited to a particular course. Program outcomes, on the other hand, stipulate the knowledge, skill, or behavior that a program’s students or participants should be able to demonstrate upon program completion. Thus, program outcomes are more general and often encompass multiple courses.

What is embedded assessment?

Embedded assessment means that assessment activities are integrated into the regular routine of teaching, learning, and research at the university. The intent is to make assessment activities regular, ongoing, and as unobtrusive and easy as possible. An embedded assessment approach also emphasizes using information you already have or collect as much as possible.

What is a rubric?

A rubric is a systematic scoring guideline used to evaluate behaviors, written work, or performance through the use of detailed, written standards.

Planning

Why is having an assessment plan important?

The assessment plan is like a roadmap that keeps you on course and helps you to avoid wasting time by going down the wrong path or gathering unnecessary data. An assessment plan also provides a focus for your efforts and establishes priorities for assessment activities.

How do I decide which data gathering method to use?

Data gathering methods should be chosen based on how well they are able to document program outcomes and program quality rather than ease of implementation. Multiple methods should be used to measure each program outcome and direct measures are always preferable to indirect measures.

Data Gathering

How much data do I need to collect and how often?

You should collect enough information to determine whether you are meeting program outcomes and meeting program goals—no more. If you cannot determine how you will use the information to measure student learning or program effectiveness, then do not collect it. The frequency of data collection depends on how you plan to use the results. In general, data collection should be regular and ongoing so that it can be used formatively as well as summatively. However, ongoing data collection does not mean collecting all data constantly. It is best to spread out data collection over time so that you collect certain pieces each semester in a regular cycle that covers multiple academic years.

My program is very large, so how do I assess the outcomes of so many classes and students?

Sampling is acceptable and encouraged for all programs. Using samples of student work already being produced and applying rubrics to evaluate the assignments used for assessment can reduce the burden on faculty.

Is one good measure of student learning for each program outcome enough?

No. Programs are expected to use multiple measures of student learning for each outcome. For example, a program might employ a capstone project, internship evaluations, writing test scores, exiting student interviews, and alumni survey data in its assessment. However, one measure can be used for several objectives; a capstone project, for example, might be used to measure knowledge in the major, research skills, and communication skills.

Why not use course grades?

Course grades, although one source of information about student achievement, are usually insufficient measures of program outcomes because 1) they generally do not identify particular areas of strength or weakness related to a specific program outcomes; 2) They almost always include factors not directly related to a program’s outcomes (e.g., class participation); and 3) Individual faculty members’ grading practices may differ greatly.

Reporting Results

What do you mean by analysis and interpretation?

Analysis helps others understand your data by describing general trends and pointing out differences and similarities. Interpretation relates data to the goals and outcomes they are supposed to measure, explores the relationships between multiple measures, qualifies, amplifies, draws inferences, and evaluates. Analysis and interpretation address questions such as the following: What do the data say about your students’ mastery of subject matter, of research skills, of writing and speaking, and so on? What do the data say about your students’ preparation for taking the next step in their careers? Are there areas in which your students are outstanding? Are they consistently weak in some respects? An attempt to address such questions through analysis and interpretation is an essential piece of any good assessment.

Do assessment results have to be used for the purpose of improvement?

In any given year, it may not be necessary or appropriate to launch a program improvement initiative based on assessment results. However, outside evaluators consistently fault assessment efforts on the grounds that assessment results are not being used to improve curriculum and instruction. While new initiatives may not be expected annually, there is an expectation that assessment results will be used to make program changes over time.

Common Concerns

Why should I want to be involved in assessment?

Finding answers to important questions about your program’s curriculum is one of the most important reasons to participate in assessment efforts. Such questions might include: How strong are our students' research skills? Can our students apply what they are learning outside of class? By the time students are seniors, are they ready for their final courses? Should we revise the sequence of our courses to enable students to learn more effectively? Answers to questions like these can help you make data-based decisions about your curriculum.

I don’t really see how assessment benefits me or my program.

When done well, the benefits of assessment are numerous but often not readily apparent in the short-term. Assessment can:

• Enhance teaching and learning.
• Inform planning and decision making.
• Improve the program and its standing in your discipline.
• Highlight program successes.
• Provide evidence of need when requesting additional resources.
• Assist with grant writing.
• Inform and assist with student recruitment.
• Assist in meeting accreditation requirements.

There isn’t time to do assessment on top of everything else.

While doing proper assessment does take time, the best assessment is ongoing and embedded in regular work processes. As an integral part of your core responsibilities, assessment activities should share their priority. Taking the time up-front to plan assessment activities will minimize the time it requires to implement them and ensure that you won’t waste time collecting information you can’t use. Also, the benefits of assessment to your program and your students make the time spent on assessment worthwhile.

Is outcomes assessment really here to stay or just a passing accreditation exercise?

While outcomes assessment is part of the SACS accreditation process it is also an integral part of UT Austin’s initiative to become a “University of the first class” and a national trend toward accountability in higher education. Increasingly, ongoing assessment is considered to be inseparable from quality instruction and the best way to demonstrate student learning and program quality.

Isn’t outcomes assessment just another way for the administration to evaluate the faculty?

No, the focus of outcomes assessment is to measure student learning across a program not to evaluate individual faculty members or students. Assessment results should be in no way tied to the promotion and tenure process.

Doesn’t assessment take control of the curriculum away from faculty?

No, the assessment process within a program is driven by the faculty. Faculty set the standards of what program graduates should know, be able to do and value. Faculty measure how well students are performing. Faculty decide what curricular or program changes are indicated by results in order to improve students’ learning within their major.

Isn’t outcomes assessment just the first step along the slippery slope to standardized testing?

No. In fact, the ability to demonstrate student learning and program quality through faculty driven outcomes assessment will help the university to resist political efforts to impose standardized testing as a way to demonstrate student learning.