The 'P' in our CAPE: Participatory Evaluation - by the community + for the community

A recent book club discussion about the challenge of adult friendships in the present day has had me thinking a lot about the importance of a sense of community.

On that note, a portion of a recent conversation between two of my favorite thinkers, Surgeon General Vivek Murthy and writer Anand Giridharadas, really struck me. Surgeon General Murthy shared that statistics show that people now are working more - “and parents are parenting more, even though they’re also working more.” He asks, “Where is that time coming from, that extra time? You put all this together and that time that is eroding is a time that we spend in person with family and friends, the time we spend for ourselves, and the time we spend for our communities.”

On the heels of a community event I helped plan, I’ve also specifically been thinking a lot about what it means to do something “by the community, for the community” (our event tagline).

What does all this have to do with the focus of today’s newsletter, participatory evaluation (aka the ‘P’ in PoP Health’s CAPE)? Let’s get into it!

What do you mean by participatory evaluation?

Simply put, participatory evaluation is monitoring, evaluation, and learning - by the community, for the community.

As more formally defined in this guide, “Participatory evaluation is not top-down or expert-led. It is a bottom-up framework that stimulates and utilizes the wealth of experiences and wisdom that participants have to create more meaningful, productive, and engaging discussions and debates.”

When it comes to the work of community coalitions and collaboratives, we want coalition members as well as community members integrally involved in every stage of monitoring, evaluation, and learning (or MEL, as it’s often called), to the point where they co-own the process, alongside their MEL team.

The idea that participatory evaluation encompasses monitoring, evaluation, and learning is key. Monitoring gets at the idea of continually examining whether things are implemented as planned, and applying what is learned to make improvements along the way. Evaluation gets at the ultimate question of “did it work”. And Learning underscores that the point of all of this is to learn together, and apply what we learn to make things better moving forward.

What are some ways to think about participatory evaluation?

As with action planning, there are hundreds of evaluation frameworks out there.

So instead of sharing or dissecting all of those, I’m going to share PoP Health’s approach to participatory evaluation + one evaluation framework specific to participatory evaluation that we have found helpful.

PoP Health’s Approach to Participatory Evaluation

Community-Driven and Co-Creative: Ensure participants share ownership of the evaluation process. Involve coalition and community members, centering and amplifying their voices in every stage of the process, from developing the plan, evaluation questions, and data collection approaches through interpretation and dissemination of results.

• Equity-Focused and Inclusive: Engage voices that have been historically excluded, emphasizing collective strengths, and maintaining a focus on upstream, root causes of health inequities. Build belonging and civic muscle through an evaluation process that helps participants develop their power to shape their world.

• Taking a Systems-Level Lens: Recognize the powerful role of policy, systems, and environmental change, and make every effort to capture and learn from the impact of changes at those levels. Ensure your evaluation process is also reflective of the dynamic behavior, complexity, and interconnectedness of systems.

• Value-Adding: Build upon ongoing activities and utilize the wealth of existing data and efforts, taking care not to waste time or resources reinventing the wheel. Identify where there is unique value to be added and focus evaluation efforts accordingly.

• Actionable: Generate relevant evidence and translate that evidence into key takeaways and concrete steps that can be taken to continually improve. Build upon assets and facilitators, address challenges, and proactively pursue transformative change that is guided by the experiences, stories, and voices of participants.

CoAct's Principles of Co-Evaluation

CoAct is focused on Citizen Social Science, which they define as follows:

Citizen Social Science combines equal collaboration between citizen groups (co-researchers) that are sharing a social concern and academic researchers. Such an approach enables [us] to address pressing social issues from the bottom up, embedded in their social contexts, with robust research methods. We aim to co-create socially robust knowledge.

They offer six principles of co-evaluation, each of which is further defined and paired with practical recommendations here.

What questions should I be asking myself about how we engage in participatory evaluation?

• What will coalition members and community members gain through the process? What new understanding, capacity, connections, resources, supports, etc. might they walk away with? ​

• How can we meaningfully co-own each stage of the monitoring, evaluation, and learning process with community members, while also being respectful of their time and other constraints?

• How can we make sure our continuous monitoring is part of a feedback loop that feeds directly into making concrete improvements?

• How do we best capture the impact that matters most to each of our audiences - community members, coalition members, policymakers, funders, and so on?

Sign up to receive future newsletters directly in your inbox at www.pophealthllc.com!

A non-cringey survey that helps coalitions understand themselves

Today, I’m going to talk about surveys.

Does that make you cringe?

I understand if it does. Surveys get a bad rap.

We’re often trying to get participants in a program or members of a community to take a survey, and - often - we don’t provide enough context for them to buy into the value of the information they’re sharing or for them to believe any changes will result from their input. To be fair, and to make matters worse, oftentimes there aren’t changes made as a result of survey responses.

There is a time and place for participant/community surveys, we certainly use them in our work here at PoP Health, and we have some thoughts on how to make them less painful and more impactful - more to come on that in a later newsletter.

But today, I’m here to talk about a kind of survey that doesn’t have quite as many potential pitfalls. A non-cringey survey, if you will.

This is a survey that coalitions and collaboratives can use to reflect on themselves. For lack of a better name, we call them partnership surveys.

These surveys are solely for members of the coalition, and the results help coalition members understand who they are and what they bring to the table, where they already have a lot of energy and expertise to move things forward, and where they may have gaps that need to be filled.

As per the latest guidance from the National Association of City and County Health Officials (NACCHO)’s, by way of their Mobilizing Action through Planning and Partnerships 2.0 process, local health departments are encouraged to begin their community health improvement assessment process with exactly this kind of survey (they call it a Community Partners Assessment).

I mentioned this briefly in the last issue of Community Threads, but we used this kind of survey in advance of our kick-off meeting with the Perry County Health Coalition in Pennsylvania earlier this month. At the meeting, we shared the survey results - which painted a picture of:

• Top action planning priorities;

• What members hope to accomplish via participation in the coalition;

• Current coalition composition, sectors and topics represented, populations touched by coalition member organizations, and suggestions around organizations, individuals, and communities to engage (either directly as coalition members or at other levels of engagement);

• Alignment between member organization focus areas and focus areas identified in the 2022 health assessment the coalition completed;

• Current and desired levels of engagement in the coalition;

• Activities and tasks members are most interested in supporting;

• Desired meeting format, frequency, and scheduling.

As you can imagine, by gathering this information ahead of the meeting and then sharing the synthesized responses and drawing out the key takeaways, we were able to facilitate a much more meaningful, nuanced discussion about where to go from here.

And since we were only surveying members of the coalition - who were already bought into the work of the coalition and wanted to actively use the results to inform their work - this survey rated pretty high on the non-cringey scale.

Have I convinced you that partnership surveys are non-cringey? Feel free to disagree with me (or share other examples of non-cringey surveys!) - drop me a line!

Sign up to receive future newsletters directly in your inbox at www.pophealthllc.com!

Leveraging the power of in-person gathering

I’ve been thinking a lot about the power of in-person gathering on the professional end as well. After a long summer vacation, this Fall has been a period of reconvening in person.

Last month, we had an in-person kick off for a DC School Behavioral Health Consortium that drove home the energy and sense of community that can come simply from sitting around the same table. I also loved the icebreaker the meeting conveners used to set the stage for our time together - they shared a collage of different images representing collaboration and asked each person, as we went around the room for introductions, to share which image resonated most with them and why. What a great way to orient the group to the reason we had come together and unearth some key distinctions and nuances in how each person at the table might approach this work. We’ve gotten used to “setting the stage” activities in virtual spaces, but they are just as valuable in in-person gatherings.

Last week, we had our second annual set of Ignite presentations and data sensemaking sessions for our work (in partnership with the Center for Health and Healthcare in Schools at GW) as the Monitoring, Evaluation, and Learning partner for an antiracist, whole child development initiative in DC schools. Although this kind of sharing of our work and making sense of data visuals could, technically speaking, be done virtually, there is something about the power of being seated in a circle and lending our full attention to the discussion at hand that is hard to recreate on Zoom. There is also perhaps something even more powerful about the “sideline” conversations that occur between partner organizations during an in-person gathering, especially one that includes time to share meals together and catch up informally, as we did on both days of this convening. One of our partners emailed us after the event, “I wanted to thank you all for such an impactful convening this week. It was really inspiring to be around such a thoughtful group of educators/advocates and to be able to more deeply connect the dots across the entire project. I was even able to leave with immediate steps to further collaborate with [two of the other partner organizations on particular projects].” This is exactly what we hope can arise from bringing people together.

And finally, later this week, we are kicking off our partnership with the Perry County Health Coalition in Pennsylvania with an in-person convening of the coalition - the first time they’ll be gathering in person since 2019! So we’re trying to be thoughtful about how we make this brief time together as meaningful as possible. One thing we did is send around a survey and conduct a few interviews several weeks in advance of the meeting. We’ll be sharing our findings from the survey and interviews at the meeting as a launchpad for further discussion - this approach allows us to get the more obvious and consensus-y ideas shared quickly, leaving more of our precious in-person time to navigate the areas where there is complexity, uncertainty, or disagreement.

None of this is to say there is not unique power in virtual gathering - here at PoP Health, we love facilitating virtual convenings and have lots of tips and tools to make that time meaningful, engaging, and productive.

It’s just that this past month in particular, we’ve also been thinking a lot about the power of in-person gathering and how we can leverage that.

Reader, curious to hear your thoughts - in your experience, what’s the unique value-add of gathering in person? And how can we best leverage the power of those (sometimes rare) opportunities to be together in the same physical space? Drop me a note!

Sign up to receive future newsletters directly in your inbox at www.pophealthllc.com!