Types of technology assessment
When introduction of instructional technology is a part of a program (e.g., critical thinking development program, in which concept mapping software is used) or constitutes an entire program (e.g., digital media service program, which include promoting multimedia project assignments, technology instruction, media lab operation, and media equipment lending, etc.), instructional technology assessment is really program evaluation that contains a focus on the use of technology.
Program evaluation is an umbrella term that encompasses several distinct types of evaluation that have different purposes. Three commonly used types are needs, process and outcome evaluation. These types of program evaluation are also conducted for assessing technology use often.
- Needs evaluation is typically used in program planning. This evaluation helps determine which program aspects or activities (e.g., use of technology) are most needed and for which population. Generally, this method is used to help develop new programs (e.g., introduction of technology) or justify existing program components (e.g., use of software). [more]
- Process evaluation explores the implementation process of a program. This approach is useful to evaluate the current processes of the program (e.g., training sessions in introducing new technological tools) and any improvements or changes that might be necessary. [more]
- Outcome evaluation is used to determine the overall effects or outcomes of the program in relation to the intended program objectives (e.g., improved learning caused by introduction of new technology). This method may not only provide whether the program objectives were met or not, but also any recommendations for improvement. [more]
Needs, process, and outcome evaluations can either be formative or summative depending upon when the program is evaluated and how the results are used.
- Formative evaluation is a study conducted during the operation of a program and provides information useful in improving implementation.
- Usually, program implementers and staff are concerned with learning how to improve their program and, therefore, interested in the formative reports from evaluations.
- Formative evaluations may provide answers to these following questions, “What changes should be made in the program’s design (needs)?”, “Which program activities are being implemented effectively and which ones are not (program)?”, “What recommendations are there for improving the program to reach or surpass all program objectives (outcome).
- Summative evaluation, in contrast, is a study conducted at the end of a program or program cycle to provide decision makers or potential consumers with judgments about the program's merit with a focus on making decisions about program continuation, termination, expansion or adoption.
- Generally, most stakeholders and sponsors are concerned with program funding and, therefore, interested in the summative reports from evaluations.
- Summative evaluations may answer questions such as, “Which population is the program most appropriate for (needs)?”, “Were all the program activities effectively implemented (process)?”, “In the end, is the program worth it (outcome)?”