When Not to Evaluate
Show me the Change
Complexity and the Art of Evaluation – Reporting Sheet
Topic: When not to evaluate
Leader: Jess Dart
Participants: Julia Davies, Sue Arendt, Scott McKenny, Maxine Schache, Carolyn Kabore, Lisa Goeman
Key Points:
• Dangers:
1. Idea is that some programs that are in the early stages of development, working out and trying different options – then predetermined indicators may ‘squash’ innovation. Evaluating a program a whole range of new ideas could stifle creativity, eg; picking winners and losers too early on
2. Danger of mandating evaluation of everything – mindset that evaluation is critical and essential. But funnily enough, this is often done at the expense of ‘diluting’ evaluation money and getting very poor quality evaluation – not everything needs evaluating
3. Focus evaluation on where learning can inform decisions – occur – multiple phase
4. Is it worth evaluating when costs exceed benefits? Eg; calculating attributable impacts to one particular program may cost more than cost of program!
5. Concept of evaluability assessment
6. Focus evaluation on what you want to know
7. Don’t do evaluation when you have nothing to learn from the answers of when you have no ideas of what you want to know. Don’t do evaluation when it is not worth it.
8. Why do we collect data that is never looked at and never used and of poor quality?
9. Potentially we could have ‘levels’ of evaluation – from very light to intensive. Focus evaluation resources where they can have most results
10. Don’t evaluate a project when its been evaluated to death
11. Need to always ask why we are evaluating – what is the purpose? Eg; when a project has really failed and everyone knows why it has not worked. What’s the point?
12. Focus more on using evaluation results.
Leave a Reply