For the last 9 months, we have had something big in the works, and we are finally ready to share it. Headlight Consulting Services has compiled our first Methods Memo to provide guidance for professionals of all levels to implement stronger CLAME practices. This particular piece’s goal is to provide evaluators practical guidance for deciding when Outcome Harvesting is a good fit and implementing the original, as well as the emergent iteration of the method. Its contents may also be relevant to organizations who are looking to establish and improve rapid feedback loops to better understand complex strategies or interventions and articulate any resulting outcomes.
When designing an evaluation, it is important to not just think immediately of the data collection approaches you might take (e.g., mixed methods including both quantitative and qualitative data) and sampling strategies (e.g., purposive, snowball, stratified random, cluster, etc.), but to first find the evaluation method that will best help structure the effort to appropriately answer the evaluation questions. Outcome Harvesting is not a one-size-fits-all evaluation method and should only be chosen if it will meet the desired aims of the evaluation.
For those unfamiliar, Outcome Harvesting is an evaluation method developed by Ricardo Wilson-Grau and colleagues to help identify, verify, substantiate, and make sense of outcomes that may be otherwise unclear or unidentified (Wilson-Grau, R. and Britt, H., 2012). Since many of the social problems practitioners are working on require input from multiple stakeholders and actors, this method leverages actors’ knowledge of each other and their work to uncover and validate what the work has led to, who has contributed, and what has been achieved. While Outcome Harvesting will not always fully detail the “how” of implementation, it will help uncover what changed and evidence of how particular actions contributed.
Outcome Harvesting can be used as both a goal-oriented and goal-free evaluation method but is a more learning-oriented approach that requires looking for both intended and unintended effects. Outcome Harvesting can be more exploratory in nature and is a good initial evaluation method for understanding and working out a theory of change for new, innovative, or complex activities. Headlight has provided a quick Evaluation Methods Decision Tree below to help provide a glimpse into potential decision-making factors used in determining the right-fit evaluation method based on the evaluation questions. This Decision Tree is not exhaustive, and we would encourage anyone making this decision to read up on evaluation methods on BetterEvaluation.org and through our new practical application Methods Memos to learn more.
The Outcome Harvesting Methods Memo is organized as use-focused modules, and we start off with a brief introduction to refresh readers on what Outcome Harvesting is, why it might be useful for evaluators, and when it is appropriate to use this method. From there, we dive into a module on what version of Outcome Harvesting an evaluator should use since Retrospective and Emergent Outcome Harvesting enable different possibilities for learning and use of findings. This subsection also includes a history of the method and comparison tables so that practitioners can make an informed decision about which version to pursue.
Once readers have selected their type of Outcome Harvesting, the next two modules dive into the step-by-step of how to implement both Retrospective and Emergent Outcome Harvesting. Module five presents Headlight’s top five recommended skills and topical areas of expertise for contracting an evaluator to conduct an Outcome Harvesting effort, that way, organizations and practitioners can ensure that they are continuing to build capacities and select those with the skills needed for implementation. The final three modules (six, seven, and eight) then get into more nuance for top tips for practitioners, implementation case study examples from Headlight’s recent work, and frequently asked questions that we have been asked by other evaluators about Outcome Harvesting that we thought would be helpful to share with others too. Finally, we have annexed four use-focused tools: an Outcome Description Template, an Outcome Identification–Sample Primary Interview Protocol, a Sample Substantiation–Secondary and Tertiary Interview Protocol, and an Evaluator Terms of Reference Outline. We hope that this Methods Memo will help inspire evaluators, project leaders, and donors alike to start or continue applying the Outcome Harvesting method where appropriate and to fuel further innovation, rigor, and adaptation in their work. If you find this resource useful, we would love to hear about what you have learned and how you have applied it in your own work. And if your organization is interested in applying this method but you need additional support from experienced evaluators, Headlight would love to support you! For more information about our services, please email info@headlightconsultingservices.com.
Comments
no comments found.