Is Developmental Evaluation a fit for all projects? (Part 2)

By Dr. Yitbarek Woldetensay, Developmental Evaluation Lead, Headlight Consulting Services, LLC. 

Introduction 

As described in our previous post, two essential questions need to be answered before deciding that Developmental Evaluation (DE) is the right fit for our project. The first question (Is DE the right fit for the proposed project?) was described in our last blog. In this blog, we will address the second question, “Is there sufficient readiness to implement DE?” and highlight critical areas to focus on in assessing an organization’s readiness for a DE. 

Is there sufficient readiness to implement DE?

Though DE originated to serve complex and innovative programs, it can only do so successfully when the organizational context is appropriate. DE requires a high level of learning readiness and a supportive organizational culture. If interested organizations are not sufficiently “ready,” the DE may fail to serve its intended purpose(s). So, before implementing a DE, a readiness assessment should be conducted and openly communicated to the client whether DE is or is not the right fit for the proposed project. 

Rationale for a readiness assessment for DE

Program evaluation, in general, is a resource-intense activity; DE requires even more substantial resource investment, given the long-term hire of a highly skilled Developmental Evaluator. Determining whether there is readiness for DE before beginning implementation can help ensure that precious evaluation resources are used appropriately. Moreover, although every organization is ready to engage in some level of learning, a developmental evaluation requires a high level of learning readiness and a supportive organizational culture. In assessing readiness for DE, the question is not whether the setting is entirely ready for DE but in what ways it is ready and how we will adapt our approach in response to the readiness assessment findings. You’ll find some key aspects of a DE readiness assessment summarized below based on the DE Diagnostic Checklist by Mark Cabaj. 

Key areas to focus on during a readiness assessment for DE

  1. Developmental situation 

In the previous blog, we described what kind of projects are amenable for DE; that is, we assessed whether the project is developmental (a project working in a dynamic and complex situation requiring new and constantly adapted approaches). After assessing a project, if you don’t feel like you’re in a developmental situation, the following alternative situations may be more applicable. 

  • Accountability Situation: Your intervention is well developed and may work in a stable environment. You may seek evaluation feedback for accountability, which aims to determine if you are implementing it with fidelity to a well-laid out and proven model. 
  • Effectiveness Situation: Your intervention is very well developed. You may seek evaluation feedback to judge the model’s effectiveness (summative evaluation). 
  • Improvement Situation: Your intervention is relatively stable and/or operating in a stable environment. You may be seeking evaluation feedback to improve the model (formative evaluation). 
  • Developmental Situation: Your intervention is developing or emerging. You may be seeking evaluation feedback to create the model. Thus, DE is a good fit for this situation. 

Once we make sure that a project is developmental; next, we need to assess the organization’s readiness for adaptive leadership capacity, learning, and evaluation. Assessing the developmental situation, adaptive leadership capacity, and readiness for learning and evaluation are incredibly helpful in guiding discussions around whether DE is suitable for the project.

  1. Adaptive leadership capacity

For DE to be a right fit, decision-makers and those supporting them must have the capacity and interest to work adaptively and with complex issues. Better adaptive leadership capacity is expected if the organization:

  • Has a history of innovation and tackling complex challenges,
  • Are comfortable with ambiguity, uncertainty, and the tension of adaptive work,
  • Are motivated to try something new and committed to a systematic innovation process,
  • Has sufficient resources to carry out its work and invest more if/when promising new avenues emerge,
  • Are willing to “learn-by-doing,” allowing the intervention to emerge over time, rather than “plan the work and work the plan,”
  • Has the flexibility and authority to change the emerging intervention to reflect new learnings and shifts in the environment, 
  • Has permission and room to make “safe-to-fail” errors and mistakes in search of what does and does not work,
  • Are more interested in learning and getting results than being perceived as “right,” and
  • Has time and patience to experiment with new approaches and generate results
  1. Readiness for learning & evaluation

For DE to be effective, organizations must be ready to engage in deep learning, reflection, and evaluative thinking. So, before starting the DE process, one should assess whether the organization is ready to embrace evaluative feedback and data to help make decisions. If the assessment concludes with high readiness for learning and evaluation, DE is the right fit. Better readiness for learning and evaluation is expected if the organization:

  • Is hungry for evaluative feedback,
  • Understands that the organization operates with cultural and cognitive biases that “shape” how the feedback is interpreted,
  • Has a history of gathering, analyzing, and making sense of data (or is fully prepared to go forward),
  • Has a culture of curiosity, inquiry, and critical reflection, 
  • Has a demonstrated commitment to “data-based” decision-making,
  • Has had positive experiences with evaluation (and evaluators) in the past,
  • Understands and broadly supports developmental evaluation,
  • Is prepared to commit time and resources to the developmental evaluation, and
  • Has someone (internal or external) in the role of Developmental Evaluator

Closing

If you’re considering implementing a DE and would like to conduct a readiness assessment using some of the guidance and prompts above, we would recommend holding a workshop with whomever you anticipate to be essential stakeholders of the DE where you can jointly discuss and share reflections on these prompts. It may alternatively be helpful to do some individual reflections and documentation of rationale against the above criteria and considerations that you could then bring to decision-makers as part of advocating for implementing a DE. Remember to refer to real-life examples, especially around adaptive leadership capacities and readiness for learning and evaluation, to ensure you will have sufficient buy-in and use of the DE for it to bring the desired value to your project and organization. This early prep work can help ensure you get a good return on your investment and that your DE is set up for success! 

To conclude, if a situation is not developmental, there is no need to assess the organization’s readiness for adaptive leadership; similarly, if an organization demonstrates weak adaptive leadership, it will not make productive use of evaluation feedback. If the organization feels that its work is developmental, that they are adaptive, and will use evaluative feedback, then DE is the right fit. 

Do you have more questions about if a DE is right for you or need support implementing a DE? If so, please email us at info@headlightconsultingservices.com

Sources 

Cabaj, M. (2014). Developmental evaluation: Diagnostic checklist. Edmonton, Alberta, Canada: Here to There Consulting. Retrieved from https://mcconnellfoundation.ca/wp-content/uploads/2021/05/Developmental-Evaluation-Diagnostic-Checklist.pdf

Gamble, J. (2008). A Developmental Evaluation Primer. The J.W. McConnell Family Foundation. Montreal

Implementing Developmental Evaluation: A Practical Guide for Evaluators and Administrators. (2019) U.S. Agency for International Development. https://www.usaid.gov/GlobalDevLab/MERLIN/DEPA-MERL/uptake-developmental-evaluation-de/practical-guide-funders

Spark Policy Institute (2014). Developmental Evaluation Toolkit. Retrieved from: https://www.alnap.org/system/files/content/resource/files/main/ReadinessAssessmentProtocol.pdf

Patton, Q. (2011). Essentials of Utilization-Focused Evaluation. Los Angeles: Sage Publications. 


Leave a Reply

Comments

no comments found.