In this Blog Salon’s first post, Maurine Knighton opens with a quote from William Bruce Cameron, “Not everything that can be counted counts, and not everything that counts can be counted.” The second half of that quote – “not everything that counts can be counted” – speaks directly to why the work of the Evaluation Lab is so timely and essential to the advancement of cultural equity in the arts. Artists and cultural workers who are deeply embedding social justice in their work are at the margins of our sector in funding and their work is made invisible by the majority of established institutions. Additionally, the work they are doing is rarely summed up by the standard metrics that funders require –– statistics culled from box office receipts and demographic surveys. Measuring change is an admirable task that will be innovated from the ground, not the air.

How do we measure the things not easily visible to the naked eye, or what I think of as the “dark matter” of arts and culture? Dark matter can’t be seen or measured, but it is believed that it makes up the majority of the mass of the universe. Similarly, a great deal of the impact of the work of art and social justice is at the cellular level and not easily measured. The measurements are there, but they usually show up in the existing history, stories, memory, and relationships that are created by and through the artists’ practices and products and held in community trust.

Standard evaluation practices attempt to measure a completed cycle of events, but social change does not happen within a delineated timeframe. At best, measuring the work of any project is more comparable to taking a small frame picture of a vast ecosystem. In a single or multi-year project / program, not enough of the picture is visible to be able to measure the long-term intentions of the work of transformative change. This concept was beautifully written about by Mondo Bizarro’s Nick Slie. Nick used the metaphor of a fallen tree providing more nutrients to the forest’s ecosystem in its death than it could in its life. Unfortunately, the forest manager sees only the short-term impact of having a dead tree obstructing the forest floor and removes it. This larger picture of the system-wide impact that Nick refers to is key to understanding where the status quo approach to evaluation falls short.

The work of the arts and social justice sector is as diverse as the communities that ground the work. Just this fact should question the one-size-fits-all approach of evaluation administered within philanthropy. I see much of the work in the arts and social justice sector as a field of experiments in human interaction and creative development, testing many different but connected hypothesis. In a more practical scenario, artists would have the ability to approach their work with a hypothetical framework and be afforded the space to talk about each step of the process, the outcomes and learning that results from the work within the window of support that has been provided. This scenario, recognizes that the development and engagement process and final product, if one is created, is not tied to the development of outcomes, but to the development of the process –the process of doing and learning and using that learning to inform more doing and sharing. Evaluation is iterative and should not be driven by philanthropy’s needs for results that speak to their bottom line and fall in lock-step with appropriated institutional language. There is an opportunity here for philanthropy to redefine their role from funder to partner.

In Risë Wilson’s blog she states, “Don’t give over your power if the terminology or tools aren’t your flavor.” I agree and would add that the process of creating language that guides our field should be rooted in practice, instead of being driven by the study of the practice by non-practitioners. This process creates a knowledge gap and disconnection from the work on the ground that is essential to understanding evaluation dysfunction. The language that we’re using –aesthetics, placemaking, metrics – is not language that is being used in the communities served by this work. Trying to put language on that doesn’t fit what you’re doing is a major obstacle to evaluation being connected to the source of the work.

There is a need to find a common vocabulary and language, and this need should be filled by those communities that are closest to the work. In the partnership framework there is an opportunity to work together to create consensus, recognizing the unique and disproportionate power relationships that are found in foundation/grantee/community ecosystem. Language is ever-evolving and always shifting, it’s not a static thing. The challenge comes in not the act of trying to find common language, but who is in that conversation. Often artists and community participants or audiences aren’t a part of that conversation.

Currently there is a lack of transparency about how measurements required of grantees are being used to support foundations’ own reporting. There is also a lack of clarity as to how evaluation can be used to mutually benefit both the grantee and the funder. Artists have little to no access to the design of the measurement or impact tools. We’re not building it together. You’re building it, you’re giving it to me, and I’m having to report using these metrics that benefit you but do not benefit me. This cycle will continue unless we can talk about it and create metrics that make sense for both of us. This is an opportunity for the development of mutually beneficial relationships and partnerships that can help us all become better at what we do.

The Evaluation Learning Lab is one of those places where collaboration, partnership, and relationships are valued. In the Evaluation Learning Lab you have multiple perspectives from funders, artists, community members, analysts, and the like, helping to inform a suite of processes and approaches designed to shift the field. This group is being proactive in the development of a practice rooted in multiple perspectives, with the intention of shifting the fulcrum of power in determining the effectiveness of artistic practice rooted in social justice. The Evaluation Learning Lab is a great example of the types of shifts that need to happen in our field. I want to thank Animating Democracy and the Nathan Cummings Foundation for their courage and insight in making this space available, accessible, and transparent.