Evaluative Focus Groups

This month SAGE MethodSpace will focus on evaluation and other types of applied research. Posts in the series and relevant posts from the archives will be found using this link. 

First-Hand Experience

In between academic positions, I spent a few years working for a non-profit organization that was part of the first class of the AmeriCorps program. AmeriCorps, intended as a domestic peace corps, was designed as a public-private partnership. With funding from the US Federal government, foundation grants, corporate and private donations, we had lots of accountability requirements. That meant, of course, attention to evaluation. 

Our services took the form of intergenerational service learning. Naturally, we needed the quantitative side to keep track of expenditures and to relate costs to outcomes. We needed to verify how many youth and elders participated, how many schools and agencies partnered. We needed to show we'd met the targets set out in our proposals and contracts. However, numbers were inadequate on their own.  To grasp success factors and problems with service delivery, we needed to understand the experiences of the youth and elder participants, the AmeriCorps members, the teachers and program staff engaged in the program. 

Focus Groups to Evaluate Collaborative Projects

Given the collaborative nature and diverse stakeholders in this program, focus groups were an ideal way to collect evaluative data. Liamputtong (2011) noted that:

Focus groups have started to gain popularity in research relating to different social groups and in cross-cultural and development research. The main argument for using them in this context is their collective nature. This may suit people who cannot articulate their thoughts easily and provide collective power to marginalised people. (p. 2)

The program being evaluated served at-risk youth and elderly people who were often in low-income or inner city areas. Teachers and staff members were often from those communities as well. Focus groups validated the worth of the work of people in challenging and often thankless jobs. At the same time, focus groups allowed these participants to offer a more complete story than we could gather from end-of-year reports or financial records, because they created a collective story. Participants involved in various capacities within the program could add, refine, or differ with each other's perspectives. 

Since I needed to observe the program in action, I conducted face-to-face focus groups in conjunction with site visits. Focus groups included collaborative partners from participating schools, senior facilities, and administrative staff. The size of the groups varied, generally between 6 and 15 participants, and lasted under one hour.

I found that the value of the focus groups was greater than the data collected for program evaluation purposes. In the groups I moderated major disagreements did not emerge, which would have required a different approach. Instead, focus groups allowed the teachers, staff, and AmeriCorps members to reflect on their work in a broader context, and to see how their respective efforts benefited others. As a result, the focus groups boosted morale and strengthened the program. 

8 Tips for Using Focus Groups for Evaluation

Here are a few tips, based on my own evaluation focus group experiences:

1. Prepare. Create a question guide, but allow time and flexibility to follow up on themes or issues that emerge in the discussion. Prepare open-ended questions that allow for elaboration on questions asked in any reporting, questionnaires, or other data collection. Articulate broad introductory questions, focused questions that ask participants to go deeper into specific matters, and summarizing questions that tie up one topic before moving to the next.

2. Develop your moderator style. Clarify your role as moderator, and stick with your role.

3. Choose a setting conducive to interaction. Find a quiet, private space. In a face-to-face group, use a circular seating arrangement. 

4. Invite productive discussion. Clarify expectations and parameters before the group formally begins.

5. Prepare to handle disagreements or conflict. Include in your opening statement your process for dealing with disagreements or conflict. Keep in mind that the participants need to know that it is acceptable for them to respectfully disagree, in order to elicit a wide range of perspectives.

6. Set the tone. Allow time for introductions if participants do not know one another.

7. Focus on the discussion. Record the focus group and/or engage someone else as note-taker.

8. Thank participants. Make sure you allow time for closure, including letting participants know how you will use the data, as well as how you will address problems they brought forward.

Liamputtong, P. (2011). Focus group methodology: Principles and practice. Thousand Oaks: SAGE Publications.