Evaluating Public Benefit

How Changing Evaluation Practice Might Help to Solve an Identity Problem

As an evaluator working in the nonprofit sector, I’m often struck by the things our society sees as inherently, obviously worthwhile, and the things it feels it needs to evaluate. For example, everyone seems to take it as given that public parks are not simply worthwhile, but are in fact a fundamental and necessary element of a good community. Although an evaluation of the value of parks might be really interesting, I’ve never seen anyone demand evidence of the impact of one before investing money. Parks have a clear, strong, commonly understood identity.

ONN’s new paper, Introducing the Public Benefit Nonprofit Sector, makes it clear that society as whole does not tend to look at public benefit organizations as inherently valuable, even though a compelling case can be made that our communities need these organizations at least as much as they need parks. I think this is a really important point. Because the term “non-profit” lumps together public benefit organizations with the very different “mutual benefit” groups, the public benefit non-profit sector is left with a serious identity problem.

This paper really got me thinking. In my work, I see this identity problem reflected in the ways in which government approaches evaluation of the nonprofit sector.  Their evaluation requirements often focus on how effectively non-profit organizations have delivered particular services or events – an approach that implies a transactional, purchase-of-service type relationship between the sector and its funders.  We see this same transactional attitude reflected throughout society.  For example, we see it in calls for the sector to be judged by the degree to which it can minimize spending on overhead.

What would evaluation look like in a world where society understood and valued the public benefit sector? Where it was seen as the essential glue that keeps communities together in difficult times, and a crucial buffer against disruptive trends in a changing world?  For one thing, impact would be framed in a much broader and more complex way. We would be exploring how public benefit organizations had enriched civic dialogue, helped volunteers feel a stronger sense of belonging, or enabled our communities to respond to emergencies. We might also, (as ONN’s paper recommends), be calling on government to gather data on the economic and social benefit of the sector at a societal level, so that we would be better equipped to gauge the impact of particular organizations at a community level.

In such a world, chances are evaluation would also be a much more inclusive process. If other sectors of society saw the public benefit sector as an essential partner, they would have a keen interest in learning from it. Businesses, universities, advocates, service users, donors and politicians would be working together, digging into thorny and complex public benefit evaluation questions with gusto.

Perhaps – and I recognize this is a subversive idea – evaluation results would address the political elements of our social issues more overtly. They might  highlight shortcomings of government policy, point out the effects of power imbalances and systemic discrimination, or call for changes in law.

The biggest difference in evaluation practice, in a world which understood and valued the public benefit sector, might be in the basic purpose to which evaluation methods were put.  Instead of being used to hold a subcontractor accountable, evaluation would be seen as a process to advance our shared understanding of how best to build strong communities. It would focus on learning and action.

Fortunately, we don’t need to create this very different kind of evaluation from whole cloth. It already exists. Participatory, critical evaluation, focused on systemic issues and grappling with complex, nuanced questions, is practiced regularly cross Ontario. ONN has highlighted examples of it through its work on the Sector Driven Evaluation Strategy.  

This gives me hope.  Although there is much work to do to changes society’s attitudes to the public benefit sector, we know how to do this more meaningful kind of evaluation already. If we diverted resources from evaluations focused on accountability, and did more of this kind of evaluation work, perhaps, we would be doing more than simply documenting public benefit.  We might in fact be helping people to understand why it is important.