Let’s talk about Developmental Evaluation!
In our last Silver Gummy Virtual Round Table this month we were fortunate to have Lana Wells, Principal Investigator, and Melissa Innes, Evaluation Lead, from the University of Calgary, Faculty of Social Work, talk to us about Developmental Evaluation.
The team has been using this evaluation approach in place of a conventional evaluation process and has seen great success.
What is the Difference?
Conventional Evaluation
- Renders definitive judgments of success or failure
- Measures success against predetermined goals
- Focused on and directed to external authorities and funders
- Emphasis = Accountability
Developmental Evaluation
- Provides feedback, generates learnings, supports direction or affirms changes
- Develops new measures and monitoring mechanisms as outcomes emerge & evolve
- Focused on providing innovators with evaluative support to make data-based decisions
- Emphasis = Learning
What is Developmental Evaluation?
Developmental Evaluation is an iterative approach to program evaluation that supports innovation, evolves with the process and is focused on learning and adaptation rather than accountability. Real time feedback is provided through evaluative rigour and learnings are generated in order to inform development. During the process the evaluator is typically embedded into the team and their role extends well beyond data collection and analysis.
Things you need to know about Developmental Evaluation
1. The process is not a recipe
There isn’t a particular set of methods or steps to follow. The timing and structure of the inquiry will depend on the situation, context, the people involved, and the foundational principle that things may have to change in order to inform development.
2. It’s more about questions than answers
Developmental Evaluations focus on developmental questions. What is being developed? What information do people need to make decisions about what is being developed? What’s working? What’s not? What needs to change? What needs to happen next? What difference is being made (if any)?
3. Relationship with the Evaluator
Traditional evaluations position evaluators outside the action to assure independence and objectivity. Developmental evaluations position the evaluator as an internal team member, integrated into the action with ongoing interpretation and decision-making processes.
4. Accountability
In the traditional evaluation process, accountability is determined after the fact and is focused on and directed towards external stakeholders. With the Developmental Evaluation format, accountability is real-time and centered on the team’s commitment to each other and their shared desire to make positive change in an area of mutual concern.
The Role of Developmental Evaluators:
- Support adaptive strategy
- Frame key challenges and underlying concepts
- Track developments
- Help to design and test small scale probes
- Document the process of learning
- Understand and navigate collaborative dynamics
- Facilitate the use of evaluation findings
Lana and Melissa’s team is using this approach with their Changing Contexts: Art of The Nudge Evaluation Plan that they are testing with the Calgary Police Service. This approach was specifically developed to change the culture of male-oriented settings (i.e., networks and spaces that are biased towards, dominated by, and/or designed for men) to advance equity, prevent discrimination and support pro-social behaviours.
Since 2020, Shift/UCalgary has been working with the Calgary Police Service to implement The Art of the Nudge (AOTN), an evidence-informed approach designed to enhance safety, equity, inclusion, and belonging within male-dominated settings. The approach draws on research related to nudges and social norms to shift culture and support more equitable behaviours. In 2022, they were awarded a federal grant by the Public Health Agency of Canada (PHAC) to further extend and evaluate the approach. The grant includes funding to support knowledge mobilization with AACP members so that other policing organizations are aware of the approach and have access to some of the tools and resources we develop.
While we know men are critical to shifting environmental cues (e.g., social norms, cultural dynamics, organizational structures, etc.) within male dominated settings, all genders have a role to play in ending gender-based violence and inequality.
What does your organization consider when it designs evaluations?
- What has worked?
- What hasn’t?
- How do you drive the evaluation conversation so that your funders aren’t setting the agenda, but rather, so that you are working together to disseminate information needed to make informed decisions and program design?
- What would an embedded evaluator in your program look like?
Want to learn more about what we do and who we support? Email us at info@silvergummy.org
