Evaluate programs

Analyzing focus group data

During data collection:

Refine your focus

If you are conducting multiple focus groups with similar participants, results from the first focus group may suggest topics to emphasize or include in later focus groups.

Reassess central questions

Based on the initial focus group, determine if central questions are still relevant. For example, imagine you begin an evaluation of an English instructional technology program for second language learners in college. An initial question you and administrators agree on is: "What is the average increase in English writing ability from this program?" During the first focus group, however, you realize that most students in the study are taking an ESL writing course in conjunction with this program. You might replace your initial question with a new one, "What benefits do second language students derive from this program technology?" Although mid-course adjustments are sometimes needed, in most cases you should adhere to the study’s purpose, continually assess if data collection and analysis are answering central questions.

Transcribe the focus groups

Consider the following questions when transcribing data:

If you hire a transcriber, explain how to format documents following your transcription rules. Be sure to check the transcription against the audiotape for accuracy. Providing transcribers with your focus group questions is also helpful.

Review transcripts

Transcribe focus groups quickly so you can resolve ambiguities while the session is still fresh. If you are conducting multiple focus groups, review your notes and transcripts to identify any additional topics you want to pursue in the next focus group by asking, "What do I still need to know or confirm?"

Record insights and summarize your reflections after each focus group

When you have important realizations during a focus group, write them down as soon as possible. If one or two participants disagree with the rest of the group, probe to understand why.

After data collection:

Develop coding categories

Not all evaluations or assessments require in-depth coding. For small studies or when you use multiple sources of data, it may be enough to identify key issues by listening to the tape and taking notes. When using multiple focus groups or relying solely on focus group data to make conclusions, transcribe and code data.

Coding speech into categories enables you to organize large amounts of text and to discover patterns that would be difficult to detect by just listening to a tape or reading a transcript. Always keep an original copy of your transcripts. Bogdan and Biklin (1998) suggest first ordering focus group transcripts and other information chronologically or by some other criteria. Carefully review all data at least twice during long, undisturbed periods. Next, conduct initial coding by generating numerous category codes as you read responses, labeling data that are related without worrying about the variety of categories. Write notes to yourself, listing ideas or diagramming relationships you notice, and watch for special vocabulary that respondents use because it often indicates an important topic. Because codes are not always mutually exclusive, a piece of text might be assigned several codes. Last, use focused coding to eliminate, combine, or subdivide coding categories, and look for repeating ideas and larger themes that connect codes. Repeating ideas are the same idea expressed by different respondents or across focus groups, while a theme is a larger topic that organizes or connects a group of repeating ideas. Try to limit final codes to between 30 and 50. After you have developed coding categories, make a list that assigns each code an abbreviation and description. [more]

Berkowitz (1997) suggests considering six questions when coding and analyzing qualitative data:

Bogdan and Biklin (1998) provide common types of coding categories, but emphasize that your central questions shape your coding scheme.

Software programs can help with coding focus group data, understanding conceptual relationships, or counting key words. They facilitate systematic, efficient coding and complex analyses. Three popular software packages for qualitative coding and data analysis are Atlas.ti and NVivo7 and XSight.

Use visual devices to organize and guide your study

You may want to use matrices, concept maps, flow charts, or diagrams to illustrate relationships or themes. Visual devices can aid critical thinking, confirmation of themes, or consideration of new relationships or explanations.

Share results with sponsors and stakeholders

Often, it is a good idea to share results with sponsors and stakeholders before you have completed your analysis. This is particularly true when doing a formative evaluation or when the sponsor wishes to make changes quickly. For example, you might share the focus group transcripts without providing any interpretation. Avoid making conclusions before you have fully analyzed the data.

Additional information

Berkowitz, S. (1997). Analyzing Qualitative Data. In J. Frechtling, L. Sharp, and Westat (Eds.), User-Friendly Handbook for Mixed Method Evaluations (Chapter 4). Retrieved June 21, 2006 from National Science Foundation, Directorate of Education and Human Resources Web site: http://www.ehr.nsf.gov/EHR/REC/pubs/NSF97-153/CHAP_4.HTM

Bogdan R. B. & Biklin, S. K. (1998). Qualitative Research for Education: An Introduction to Theory and Methods, Third Edition. Needham Heights, MA: Allyn and Bacon.
Seidel, J. V. (1998). Qualitative Data Analysis. Retrieved June 21, 2006 from: http://www.qualisresearch.com/QDA.htm

Page last updated: Sep 21 2011
Copyright © 2007, The University of Texas at Austin