Programs & Projects

Advancing the Use of Developmental Evaluation at USAID (DEPA-MERL)

Overview

Developmental evaluation (DE) was created to evaluate innovative programs that operate in complex environments and are thus expected to adapt over time. The Developmental Evaluation Pilot Activity (DEPA-MERL), developed under the U.S. Global Development Lab’s Monitoring, Evaluation, Research and Learning Innovations (MERLIN) program at the United States Agency for International Development (USAID), is testing the effectiveness of DE in the USAID context. The DEPA-MERL consortium consists of Social Impact (prime), Search for Common Ground (Search), and the William Davidson Institute (WDI) at the University of Michigan.

As part of the consortium, Social Impact and Search are implementing DEs while WDI is serving as an evaluator to assess the effectiveness of this approach in the USAID context.

The WDI team seeks to answer the following three research questions:

  1. How does DE capture, promote, and enable the utilization of emergent learnings in support of ongoing programming in a complex system, such as in the USAID context?
  2. What are the barriers and enablers to implementation of DE in the USAID context?
  3. What do key stakeholders consider to be the value (added or lost) of conducting a DE compared to a traditional evaluation approach?

The Work

1. Family Care First in Cambodia (Nov 2016 – Mar 2018)

DEPA-MERL conducted a DE with Family Care First (FCF) in Cambodia, in service of FCF’s goal of increasing the number of children living in safe, nurturing family-based care.

  1. A Study of the Family Care First in Cambodia Developmental Evaluation
    Executive Summary | Full Report | Annex
    This report provides results from an in-depth mixed-method analysis of a 15-month DE that was conducted with Family Care First in Cambodia.
  2. Final Report from the Family Care First Developmental Evaluation
    Full Report
    Developed by Search for Common Ground, this report documents the implementation of the DE with the Family Care First Initiative in Cambodia.

2. US Global Development Lab (Feb 2017 – Dec 2018)

The Sustained Uptake DE was conducted in service of the Lab’s mission to source, test, and scale development solutions. The 22-month long DE helped several of the Lab’s teams to collect, analyze, and disseminate learnings regarding the uptake of the innovations that these teams seek to promote within and beyond USAID.

  1. A Study of the Sustained Uptake Developmental Evaluation
    Executive Summary | Full Report | Recommendations
    This report shares results from an in-depth mixed-method study of a 22-month DE that was conducted with the US Global Development Lab.
  2. Top Tips for Conducting Developmental Evaluation at USAID
    This one-pager provides guidance on how to manage barriers and promote enablers that influence DE implementation.
  3. Developmental Evaluation: How Barriers and Enablers Emerge Over Time
    This document shares the results from an in-depth qualitative analysis of the barriers and enablers that surfaced during implementation of the Sustained Uptake DE.
  4. Final Report from the Sustained Uptake Developmental Evaluation
    Full Report
    Written by Social Impact, this report documents the implementation of DEPA-MERL’s DE pilot with the Global Development Lab.

3. Bureau for Food Security (Dec 2018 – Sept 2019)

USAID’s transition from Bureau for Food Security (BFS) to the proposed Bureau for Resilience and Food Security (RFS) offered a unique opportunity for the Bureau to use DE to examine its knowledge management practices and improve its processes and systems around evidence-based decision making. While the DE ultimately did not fully launch, key lessons from the attempt were documented.

  1. Key Lessons from an Attempted Developmental Evaluation Pilot 
    These lessons explore the various factors that impacted the DE start-up and buy-in processes. They are broadly applicable and can be used by USAID or other interested practitioners to decide whether or not to conduct or participate in a DE.
  2. Final Report from the Bureau for Food Security Developmental Evaluation
    Written by Social Impact, this report documents the attempted implementation of DE with BFS.

4. Digital Strategy (2020-2021) 

In early 2020, USAID launched its first-ever Digital Strategy in order to align the Agency’s vision for development assistance with the world’s evolving digital landscape. USAID’s Innovation, Technology, and Research (ITR) Hub, formerly the U.S. Global Development Lab, is responsible for leading the strategy implementation. DEPA-MERL launched a new DE in mid-2020 to support up to four of the Strategy’s 15 implementation initiatives by providing insights into how the initiatives work within their teams and with each other as well as timely input for decision making. For the first time at USAID, the Developmental Evaluator is being embedded remotely into the initiatives due to the COVID-19 pandemic.

The WDI team also conducted an analysis across the three DE pilots that occurred during the first five years of the MERLIN award. The report, Advancing the Use of Developmental Evaluation: A Summary of Key Questions Answered during a Multiyear Study of Developmental Evaluations Implemented at USAID, shares findings and lessons learned across the experiences to facilitate learning from the implementation of DE in the USAID context.

In addition to the pilots, the consortium also provides guidance for organizations, managers, and evaluators that seek to implement the developmental evaluation approach. Available resources include:

Please find additional case studies on the use of developmental evaluation including in non-DEPA MERL pilots here. Each case study provides background on the programming, the DE design and implementation approach, the suitability of DE, and lessons learned for implementation.

Using data from the first three pilots, WDI also published a paper in the Journal of MultiDisciplinary Evaluation. The paper explores how DE theory is used in practice and aims to expand the evidence base and strengthen future DE implementation.

This article is made possible by the generous support of the American people through the United States Agency for International Development (USAID). The contents are the responsibility of WDI and do not necessarily reflect the views of USAID or the United States Government. This article was produced by the William Davidson Institute at the University of Michigan in collaboration with Social Impact under the Developmental Evaluation Pilot Activity (DEPA-MERL), Contract Order Number AID-LAB-C-15-00002, funded by the U.S. Agency for International Development.

Back to Top