Evaluate programs

Document analysis

Overview

Document analysis is the systematic examination of program documents such as mission statements, training materials, policy and procedure manuals or client instructions in order to identify program needs and challenges or to describe the program. The focus of the analysis should be a critical examination, rather than a mere description, of the documents. The analysis should include questions about the purpose of the document; how the program staff or clients are using it, and how it is (or is not) contributing to program goals.

Document analysis works best when the purpose is to gain insight into program activity or service. It can help you to recognize patterns you might miss if you examine one client experience at a time.

Limitations of document analysis:

Documents or materials may be incomplete or missing
Data is restricted to what already exists
Does not evaluate current staff or client opinion, needs, or satisfaction

Resource requirements

Minimal resources are required, primarily involving the time to select and analyze documents. You can complete the analysis without involving clients, or interrupting regular program operations. Experience or training in content analysis is helpful. Analyzing program documents requires a low to medium time commitment, depending on the number of documents you want to examine. [more]

Plan your document analysis

STEP 1. Describe the context

Include the age, majors, educational background, motivation level, and skill levels of students/clients. Also consider central goals of the program, your ability to implement changes, and how the organizational context impacts your program. A worksheet is available to help you document your program context.

STEP 2. Identify stakeholder needs and develop central questions

Identify what is most essential for clients, program staff, and any organizational priorities that impact your program. Central questions, informed by these needs, specify what you want to learn from an evaluation. A worksheet is available to help you identify stakeholder needs and develop central questions.

STEP 3. Determine the purpose of the document analysis

Because it is too cumbersome to examine every aspect of a program at once, start with clear goals about what you would like to learn and narrow your focus. Whether you analyze documents to immediately make program changes or as part of an overall evaluation will determine your focus and what documents to review. A worksheet is available to help you develop and refine your study’s purposes.

STEP 4. Determine how you will use the results

How you intend to use results should also guide the focus of your document analysis. If analyzing a particular document will not guide program content or operations, choose a different document or consider another data gathering method. A worksheet is available to help exemplify how to use results after determining the purpose of a study.

STEP 5. Develop document analysis criteria

Establish clear criteria before you analyze documents. How deeply you analyze documents depends on your central question(s). Make sure you establish clear criteria for ratings such as "none," "little," "medium," or "extensive" and concretely define the relative importance of different criteria.