Contents:

Definition of "Administrative Unit"
Outcomes-Based Assessment

Definition of "Administrative Unit"

"Administrative Units" are defined here as those director-led administrative entities that provide “administrative and educational support services” as stated in Comprehensive Standard 3.3.1. While this could effectively include every administrative unit on campus (because one of the University’s primary missions is to educate students), SACSCOC focuses this standard on student learning. Thus, those administrative entities that directly support student learning and are director-led are included in this definition of Administrative Units.

The administrative units that have been identified as needing to prepare assessment plans are:

• Provost’s Office – Admissions, and International Office
• Employee and Campus Services - Units with classroom and teaching laboratory focus within Facilities Services, Capital Projects, Project Management and Construction Services
• Information Technology – Focus on IT related to student learning within ITS and the schools/colleges
• Student Affairs – Career Exploration Center, Counseling & Mental Health Center, Dean of Students Housing and Food Service, Learning Center, Multicultural Information Center, Office of the Registrar, Recreational Sports, Student Financial Services, Texas Student Publications, Texas Union, University Health Services, and University Honors Center & First Year Interest Groups

Outcomes-Based Assessment

Comprehensive Standard 3.3.1 states that: The institution identifies expected outcomes for its …administrative and educational support services; assesses whether it achieves these outcomes; and provides evidence of improvement based on analysis of those results.

To elaborate, outcomes-based assessment is a continuous improvement process. One of the consequences of this approach is a continuous effort to show that outcomes established for an administrative unit are being achieved and that can be demonstrated through information gathering, determination of areas of needed improvement, making changes to implement those improvements, and then showing that those changes were effective. This “closing the loop” approach takes time, but at accreditation review time, the program evaluators will want to see evidence that assessment processes are in place and that the program has cycled through this loop one or more times at least.

How does this approach apply to administrative units? The key questions that any administrative unit must ask itself about why it exists are:

The answer to these two questions is captured in a mission statement. Other questions to be addressed are:

The "Language" of the Process

Mission Statement.The statement identifies for constituents external to the unit the purpose of the unit and describes why it exists (purpose). It usually includes what clients’ needs are being met and the services provided to meet those needs. The organizational unit’s mission statement should be consistent with the mission statement of the University.

Mission statements play a central role in defining the enduring character of an institution and the educational and administrative units within that institution. When properly defined and communicated, mission statements capture the unchanging purpose for the institution and its educational and administrative units and their primary defining character. When effectively implemented the mission statements provide guidance in all activities for each of these units and serve as a compass in setting direction and priorities through the years. All employees of the institution or any of the educational and administrative units should be familiar with the institution’s mission statement as well as that for their unit, and its influence should be pervasive.

Mission statements also play a central role in accreditation. As noted in "The Role of Institutional Mission and Institutional Effectiveness in SACSCOC's Reaffirmation of Accreditation (pdf)," institutional, educational unit, and administrative unit missions play key roles in continuous improvement through the outcomes-based assessment approach. Professional program criteria also typically require that the educational program must develop an assessment program which measures educational outcomes for consistency with the mission statement of the institution within which it is housed.

Mission statements are widely acknowledged in the modern business community to play a central role in the development of lasting corporate success. For example, the RAND Corporation has proposed criteria for a good mission statement. These criteria suggest that a good mission statement should be (a) clear and differentiating, (b) inspirational and motivating, (c) relevant and realistic, and (d) pervasive and enduring. There is evidence that top performing and enduring companies develop a culture which is characterized by a high degree of internal alignment with the mission and core values.

Goals: Broad statements about what the organizational unit is trying to accomplish to meet its mission (including major issues that are faced). The goal statements are general few in number and describe the primary things that the unit is trying to accomplish.

Objectives: Specific results toward which effort is directed to achieve the each goal. Objective statements further define the goal and serve to identify specific activities that take place to achieve each goal. By monitoring the objectives through the use of performance indicators, the unit can identify how well it is achieving its goals.

Note that there are operational objectives and strategic objectives. Operational objectives identify the ongoing objectives that need to be met to achieve the institutional or unit mission. The objectives are periodically reviewed for relevance, monitored systematically, and linked to resource allocation and annual budgeting processes. Strategic objectives, on the other hand, identify the major challenges and/or change opportunities for the next three to five years in order to remain and/or become competitive and achieve the mission (results from strategic planning process).

Performance Indicators: High-level numeric or qualitative measures that that will be used to monitor progress toward achieving the objective. These are generally metrics that a unit is already monitoring (e.g., time to completion, number of applications processed, customer satisfaction ratings, dollars raised, number of complaints, etc.). The metrics serve as indicators of how well the unit is meeting its objectives.

Administrative Unit Framework

The relationship for the mission, goals, objectives, and performance objectives is depicted in the pyramid below, i.e., the performance indicators support the objectives and must be achieved if the objectives are to be achieved. Likewise, the objectives must be achieved if the goals are to be achieved, and achieving the goals means the mission is achieved.

Implementation of this framework of mission, goals, objectives, and performance indicators starts with addressing the question, “what goals do you need to achieve your mission?” Remember that goals are broad statements about what the institution is trying to accomplish to achieve its mission, and four to six goals might be developed. For each of the, the next question is “what objectives are related to each goal?” This may require that you define the key components related to each goal. For example, teaching will undoubtedly be part of the mission of the institution, so a goal related to teaching might be to develop an excellent learning environment.

How does an institution achieve the goal of developing an “excellent learning environment?” While this depends on the definition of “learning environment”, i.e., is it limited what happens in the classroom only or does it include the classroom facility itself, the libraries, study space, dormitories, and so forth, one would expect that excellent teaching, appropriate facilities and resources, and so forth would be included in the scope of this goal. And each of parts of the learning environment can be incorporated into an objective, objectives or specific results that would need to be achieved if the goal is to be achieved. For example, excellent teaching could be expressed as one objective and provide facilities, equipment, and necessary resources as another.

How would you know if this objective is being accomplished, what performance indicators could be established that would point specifically to excellence in teaching? Two performance indicators might be “hold six or more teaching workshops/seminars annually” and “XX% of graduating seniors rate the overall quality of the preparation they have received to be excellent or good.” Note that these are quantitative indicators, the indicators are ones that relate strongly to the objective, and achievement of these indicators contributes to achievement of the objective.

Other information that will be needed include the current level of performance, the goals or targets set, the source of the data to be collected, the individual or office to collect and analyze the data, the timeline for collecting and processing the data, and the office responsible for using the data to improve the unit.

An implementation plan must be put into place to effect data collection, data analysis, and acting on the results of that analysis, and this plan can be viewed as a Plan-Do-Check-Act process in the diagram below. This process was originally developed by Walter Shewhart of Bell Laboratories in the 1930’s and was later adopted and used effectively by W. Edwards Deming from the 1950’s. Thus, the process is referred to as “the Shewhart Cycle” and “the Deming Wheel”.

Approach

The approach of the Plan-Do-Check-Act quality improvement cycle can be viewed as a planned sequence of systematic and documented activities focused on improving a process. Notice the focus on process – a set of steps or tasks by which an objective is achieved. Improvements can be effected by improving the process itself and/or by improving the outcomes of the process and doing so in four steps – Plan, Do, Check, and Act.

1. Plan (the change)
   This can include: (a) selection of the change or identification of the need to implement change; (b) reflection on and interpretation of relevant information concerning the existing process drawn from as wide a range of sources as possible and including information from clients and stakeholders; (c) definition of the current process and the opportunities for improvement; (e) planning of how you will monitor the progress and the effectiveness of the change; and (f) documentation of your goals and objectives i.e., what improvements/changes do you expect to see?
   
   As part of this step, one should as such what, who, how questions such as: (a) what data indicate a change is required, what change is to be made, how do you know the planned change is appropriate and what other alternatives are there, what sequence of steps is needed to implement this change, and what will you do about unexpected problems; (b) who will be responsible for carrying our each step, who will need to be consulted, and who will the change affect; (c) how long will the change take and how long will each step take, how will you know you have completed each step, how will you monitor the change’s effectiveness and the benefits of the change, how will you monitor and track the progress of the change, how will you collect, review and act on information?
2. Do (implement the change)
   In this step, carry out the change or new practice and document the implementation activities.
3. Check (monitor and review the change)
   This can include: monitoring the progress and effectiveness of the change according to your plan; recording of observations and results (planned and unexpected) in comparison with the original data or the project goals, measures and objectives; and studying the results - what did you achieve what did you learn? Checking can go on continuously throughout the whole improvement cycle.
4. Act (revise and plan how to use the learnings)
   Ask questions such as: (a) what will you do with the learnings – adopt them, abandon them, run them through another PDCA cycle again to test; (b) what did the information you collected tell you about the effectiveness of the change; (c) what can be done to improve the process further; (d) how can the change be refined; and (e) what lessons have you learned that can be applied elsewhere and how can these lessons be communicated?

Thinking of this process improvement exercise as a cycle or wheel conveys the thought that one can improve a process through several successive cycles and implement continuous improvement. That is indeed what is intended.

Application Examples

An example to illustrate the Plan-Do-Check-Act cycle can be taken from Gloria Rogers’ workshop presentation “Administrative Effectiveness: Measuring What Matters Most” in which she used Rose-Hulman Institute of Technology’s mission, goals, and objectives.

Rose-Hulman’s mission is “to provide students with the world's best undergraduate education in engineering, mathematics, and science in an environment of individual attention and support.” Its goals to achieve this mission are:

• Goal 1: Recruit highly qualified students, faculty, and staff
• Goal 2: Develop and excellent learning environment
• Goal 3: Foster the personal and intellectual developmen of the campus community members
• Goal 4: Instill in our graduates skills appropriate to their professions and life-long learning
• Goal 5: Provide resources to support the Institute mission

Objectives within these goals that are administrative unit focused include, but are not limited to, the following:
Objective 2.3. Provide facilities, equipment, and necessary resources.

Objective 2.4. Promote opportunities for understanding different perspectives by increasing the diversity of the R-H community.

Objective 5.1 Ensure that resources allocation is consistent with the Institute’s academic mission.

Objective 5.3 The endowment per student should increase by a minimum of 2% per year over the rate of inflation for any 10 year rolling period.

The dashboard indicators for these performance indicators show that indicator 2.3.1 is at or above its target and that performance has increased since the last measurement period, that indicator 2.4.3 is up but no target has been set, that indicators 5.1.5 and 5.1.6 are at or above their targets and that they have remained steady, and that Objective 5.3 is far below its target and has not changed.

These goals were established through a comprehensive process involving multiple constituents of the Institute (faculty, staff, students, alumni, Trustees, and industry and business partners). The objectives were also established along with the performance indicators, the targets for those indicators, the sources of data to be used to determine the current level of those indicators, the timeline for checking progress, and the individual/office responsible for each objective. All of this was done as part of a PDCA process with goals, objectives, targets, etc. being refined and fine tuned until they were deemed satisfactory. Some of the goals may change over time but probably slowly. Some of the objectives may change and probably more often as they are refined or replaced by more relevant objectives. Targets should be pushed higher and become more challenging to meet as the institution improves its operations over time. Thus, the framework itself for continuous improvement of the institution is reassessed over time.

Each objective is most likely achieved through implementation of a process or processes, and those processes can be improved over time so that the objectives are achieved or are achieved with greater success than before. Here again the PDCA process comes into play. For example, it was noted that there was no target for Objective indicator 2.4.3. A Plan could be developed for determining what a target should be, information gathered from student surveys in the Do step, and the “test” target Checked, and the target implemented as an Action step if it is providing useful information to the institution or, if it is not, another target selected in the Plan step and the process repeated until a suitable target is found. For Objective 5.3 the problem is that the target is not being met, so the focus of the Plan is to develop a fundraising program to increase endowments, to implement that program in the Do step, to check fundraising progress over time in the Check step, and revise the fundraising program if progress is not as anticipated and repeat the process or to declare victory if the fundraising goal is achieved.

This process probably sounds very familiar by now and is essentially how problem solving is done in the various units on campus. While this example for Rose-Hulman is at the institutional level, hopefully it is clear how the Plan-Do-Check-Act process can be applied to the unit level.