When it comes to evaluating a national research initiative with many projects and many partners, what’s most important?
That’s what Abacus evaluators, housed in CHER, looking at RADx® Underserved Populations (RADx-UP), share in two new publications. (RADx-UP was created by the National Institutes of Health to ensure that all Americans have access to COVID-19 testing, with a focus on communities most affected by the pandemic.)
What’s most important isn’t surprising, but it’s backed up every time they survey project teams.
The two key takeaways: trust is central and flexibility matters.
Shelly Maras, Ph.D., an Evaluation Research Scientist in the School of Medicine, and the first author on both papers, shared insights on the findings. To start, she said that the two publications really have two audiences in mind.
Helping evaluators, sharing lessons learned
The first paper covers how RADx-UP developed their evaluation plan. It’s meant for “other researchers or evaluators” who are creating “qualitative plans for large, national initiatives.” The research team shares the four phases to their development plan and offers lessons learned.
According to Maras, “there’s no one-size-fits-all” way to create an evaluation plan. “You have to think about all the people involved,” she said.
With that in mind, the paper shares lessons learned that others can use “as a starting point.” Maras says they can see what RADx-UP did and “what they would have changed.”
While it’s not meant as a template, the paper does offer specific lessons for others to consider.
For example, Maras et al. emphasize the overall need for building relationships and clarifying communications and expectations. They also share specific lessons learned about methods to use (or avoid) depending on the needs of the evaluators.
There are four clusters of lessons shared:
- Collaborations and approvals.
- Sampling and interview guide creation.
- Data collection and data analysis.
- Communication and dissemination.
Lessons and recommendations are also shared in an easy-to-share table.
The how-to paper, “How to Develop a Qualitative Evaluation Plan for a Complex National Intervention: Key Steps and Reflections from the RADx-UP Program,” published in The Qualitative Report, is available open access.
Hearing from projects, emphasizing trust
The second paper shares findings from interviews the evaluation team did with representatives of 13 of the RADx-UP projects. The research team heard from academic and community partners from almost all of the 13 teams, with 24 total responses.
Maras said that this second paper was for “any researcher doing community-based participatory (CBPR) research or community-engaged research.”
She shared that community-engaged research is often a step toward full CBPR. CBPR is “very immersive,” with partners working together “start-to-finish” on projects, Maras said. She added that community-engaged research has part of that approach, but community partners may not be as “embedded” in the research aspects.
In the responses from RADx-UP projects, two major findings stood out: trust and adaptability.
Building trust in communities came up again and again from the project members RADx-UP interviewed. As one community member said, “Trust is a big thing” because of the medicines (Covid testing) and “the history we have with the U.S. and doctors.”
Maras summarized the need for adapting to communities: “Listen to community needs.” She added that responding to community needs sometimes means changing “research strategies.”
Maras emphasized that the interviews made it clear that researchers shouldn’t limit themselves to providing “just one thing.” Teams should be ready to change focus (or add a focus) if community members share a need.
The qualitative findings paper, “Qualitative Evaluation of RADx-UP Projects Addressing COVID-19 Testing Disparities Among Underserved Populations,” published in American Journal of Public Health, is available full-text online.
What’s next?
The RADx-UP evaluation team is still gathering and analyzing data. As they have more findings, you’ll find them presenting posters and publishing papers throughout the year.
Findings from more teams agree with the findings in these two papers. This suggests that the RADx-UP projects are doing what works. Their work made a difference across the US during the pandemic.
And by sharing evaluation methods and findings, they’ve committed to continuing to share the lessons they’ve learned.