Technology assessment process
Instructional technology assessment involves iterative cycles of planning, gathering data, and reporting.It is best to plan the assessment at the same time that the technology introduction is being coordinated, so that you can take necessary steps such as collection of baseline data.
All program assessments follow a planning, gathering data and reporting results process consisting of the following nine steps:
|STEP 1||Describe the technology and context||
Describe the instructional technology to be used in terms of its function or purpose, expected effects, and resources needed. Also, describe the instructional context, such as the program or course, learner characteristics, and learning objectives in relation to the technology.
|STEP 2||Identify stakeholders and their needs||
Stakeholders are the individuals and organizations involved in developing and supporting the technology, those served or affected by the technology, and the intended users of the assessment. The needs of stakeholders generally reflect the central questions the stakeholders have about the instructional technology.
Determining stakeholder needs helps to focus the assessment process so that the results are of the greatest utility.
|STEP 3||Determine the assessment purpose||
Identifying a clear purpose helps determine how to conduct the assessment. Three general purposes for instructional assessments are to gain insight, change practices, and/or measure effects.
|STEP 4||Identify intended uses||
Intended uses are the specific ways assessment results will be applied. They are the underlying goals of the assessment and are linked to the central questions of the study.
|STEP 5||Create an assessment plan||
The assessment plan is a detailed description of how the assessment will be implemented that includes identification of the resources available for implementing the plan, what information is to be gathered, the research method(s) to be used, a description of the roles and responsibilities of the stakeholders, instructor and/or assessor and a timeline for accomplishing tasks. [more]
|STEP 6||Gather data||
Data gathering focuses on information acquisition that
will convey a well-rounded picture of the instructional technology
|STEP 7||Analyze data||
Data analysis involves identifying patterns in the data, either by isolating important findings (analysis) or by combining sources of information to reach a larger understanding (synthesis), and making decisions about how to organize, classify, interrelate, compare, and display information. These decisions are guided by the questions being asked, the types of data available, and input from stakeholders.
|STEP 8||Make conclusions and recommendations||
Conclusions are linked to the evidence gathered and judged against agreed-upon standards set by stakeholders. Recommendations are actions for consideration that are based on conclusions but go beyond simple judgments about efficacy or interpretation of the evidence gathered.
|STEP 9||Report results||
Factors to consider when reporting results, or dissemination, include tailoring a report content for the audience, explaining the focus of the study and its limitations, and listing both the strengths and weaknesses of the technology. [more]
Adapted from the following sources:
Center for Disease Control Evaluation Working Group. Steps in Program Evaluation. http://www.cdc.gov/eval/steps.htm Retrieved June, 30, 2006.
Joint Committee on Educational Evaluation, James R. Sanders (chair). 1994. The Program Evaluation Standards: How to assess assessment of educational programs, 2nd ed. Sage Publications: Thousand Oaks, CA.
Popham, James W. 1993. Educational Evaluation, 3rd ed. Allyn & Bacon: Boston.
Taylor-Powell E., Rossing B., Geran J. 1998. Evaluating Collaboratives: Reaching the potential. Madison, WI: University of Wisconsin Cooperative Extension.
University of Washington, Office of Educational Assessment. Evaluation Planning Guide. http://www.washington.edu/oea/evaluatn.htm Retrieved March 26, 2004. <Note: site no longer available>