Blog

More useful evaluation for nonprofits? Yes, we can!

In 2015, ONN took on a project to look at the systemic issues of evaluation in Ontario’s nonprofit sector and design potential solutions to help us get to more useful evaluation. This framing led to the development of our Sector Driven Evaluation Strategy work, which to date now includes a resource treasure chest designed to help identify pain points experienced by nonprofits, and move forward with your stakeholders to create an evaluation process that can work better for your organization.
To tell the story of our work, we’ve created a visual timeline of our project with highlights.

As the two-year project draws to a close, we want to share some of our own evaluation findings. We recently surveyed the network to help us get a sense of the ways in which our work has been viewed, shared, and used. We hope this information is useful to you when considering ways to use our materials to support your work.

How have our resources been used?

We know there are a lot of great resources out there, but often times they can get lost among the many other great resources that exist.
Our survey results showed that the top three ways in which our materials have been used or might be used in the future were:

To improve my own evaluation knowledge or practices (78%)
To engage with funders, grantees, or partners on my evaluation needs (48.6%), and
To get more people in my organization interested in evaluation (40.4%)

What we heard

While those numbers tell part of the story, we also wanted to share some specific examples of use, in a “What we heard” format.
Developing an evaluation strategy

Our organization is working towards being more effective with our evaluation processes. The materials have been very helpful in assisting our strategy.

Improving relations with stakeholders

Used resources as part of internal training materials for staff, shared resources with staff who work with agency partners on evaluation (we are a funder).

Helping to write grants

To assist in preparing applications for grants or government programs
6 Simple Tips for Communicating About Impact. This document is invaluable because it provides practical advice for a variety of purposes, addressing community, writing proposals etc.

Developing or strengthening relationships with others on evaluation

Keeps me in touch with others thinking about evaluation
To share with network members to start a discussion about the struggles with evaluation and find potential solutions/next steps
Deeper evaluation of effective practice will support funding and resource requests and help direct program improvements and new offerings. ONN resources will help start the discussion with staff (it’s not just coming from me!).

Training or Skill development

“Five important discussion questions” has helped me with different language to use in preparing evaluation workshops for the non-profit sector.
Reading the evaluation blogs and the evaluation resources have helped me understand the various evaluation approaches and to understand how I might apply them within my organization.
Materials have been used to guide discussions on evaluation and to generate knowledge sharing among staff who are members of an evaluation project team for a non profit sector agency
The discussion questions will be used at monthly meetings to keep the team thinking about evaluation.

Understanding the issues

Has helped understand the complexity of evaluation within a nonprofit organization (eg mission-related evaluation vs. operational, financial) and the various levels of inquiry (outputs through to impact)
The infographics help to frame up evaluation in a way that is accessible to those of us who don’t have a social science background but want to understand evaluation in order to learn how and when it is most useful.

Responding to government requests

We have used summary text about ONN’s position on evaluation in responses to government requests for feedback on programs and issues.
We see value in being able to point to a trusted organizations stance on evaluation – so more principles, or content that is worded in a way that we can point to and say “We endorse/recommend the stance the ontario non-profit sector has made on xyz….”

What has ONN learned?

When we started out, one of our key goals was to start conversations about evaluation. Our findings suggest that we have certainly achieved that goal. Another important learning came as a result of our literature review and highlighted the power in naming the issues, even if the solutions are not yet readily apparent. Being able to identify and name the reasons why evaluation wasn’t working resonated.
Finally, we learned there is a hunger for ways to use evaluation that is relevant, engaging, and actionable. Our six factors that lead to useful evaluation was another conversation starter and a useful frame for breaking down the underlying relationship factors that truly matter in an evaluation process.

What we need to do next

We understand there are many demands in the sector that affect people’s time and resources. Our own evaluation shows there is still work to be done to disseminate our materials more broadly to the sector and spread the word that nonprofits can help drive evaluation strategies. Our resource treasure chest will continue to be accessible online and we encourage you to help us share the work to your own networks and teams and report back if and how it’s useful.

Our vision

We also want to re-share our 2020 vision for evaluation. Our work is but one small part in creating a system that makes it easier, more rewarding, and less stressful for nonprofits and their partners to do meaningful evaluation work. We hope the conversation will continue and, with any luck, we’ll have an evaluation system that:

leads to action more often for more purposes
addresses needs and questions that are important to a range of stakeholders
is planned, conducted, and shared in a more collaborative way; and
is used when and where it can help the most.

Final thoughts

Lastly, we’d like to thank everyone who made the time to chat with us, share their experiences, and try out resources. It’s been a great experience to have met with so many different people across Ontario and to learn about the interesting evaluation work that is taking place. Many thanks as well go to our advisory committee and to those who contributed feedback and helped to shape the creation of our resources and other materials. We couldn’t have done this work without you.