At the October Grantmakers in the Arts (GIA) conference, artist Rosten Woo described the Vendor Power! project, a poster/brochure initiated by the Center for Urban Pedagogy (CUP) and designed by artist Candy Chang to make comprehensible New York City’s most commonly violated street vending rules which are buried in hundreds of pages of impenetrable bureaucratese.  For thousands of vendors whose first language is not English, the Vendor Power! poster became an essential tool, directly helping them to understand their rights, avoid fines, and know how to respond when approached by police. Woo reported with satisfaction that, following CUP’s distribution of 10,000 posters, the Dept. of Consumer Affairs seized the poster’s power to address a longstanding institutional problem and printed another 10,000. Here the system took action to change a problematic practice.  If only evidence of change was always so clear!

Animating Democracy savors examples such as this and opportunities such as the recent Grantmakers in the Arts conference to learn and share with others about outcomes of arts and social change work that can elude easy measure. So, at GIA, we were delighted to be joined by 70 or so kindred spirits who participated in our Evaluation Test Kitchen session and offered feedback on two in-progress frameworks evolving out of the Evaluation Learning Lab.

One framework they tested was the Values Guide to Evaluating Arts & Social Change. It addresses the fact that common evaluation can fall short of being relevant, ethical, equitable, and realistic when it comes to assessing arts and social justice work. It proffers a set of values (congruent with the nature of the work itself) that should guide evaluation design and implementation—reciprocity, shared learning and understanding, context, equity, appropriate metrics, and right-sized expectations, among others—as well as related indicators of responsible evaluation. 

The second is a framework of Aesthetic Characteristics for Social Justice-Oriented Art (see also Chris Dwyer’s blog) that addresses the challenge that conventional aesthetic standards are not always well suited to discuss and assess the aesthetic/artistic qualities of arts and social change work. The Test Kitchen served up artistic qualities that are important in effective arts and social justice work. Stickiness, for one, is about memorable aesthetic take-aways, i.e. to influence social change, art inspires people over time and repeatedly. The Vendor Power poster has a stickiness factor!  Other aesthetic qualities include: disruption, authenticity, communal meaning, porosity, risk-taking, emotional response, and sensory experience.  The framework describes each quality’s meaning, importance in relation to social justice, as well as questions that can help grant panelists and other sector partners assess if and how effectively that quality is present in the work. Both frameworks received useful critique and general appreciation and we still welcome feedback from anyone inclined to dig in (see below).  

Following the conference, Animating Democracy held its second Evaluators Circle, inviting 10 Los Angeles-based cultural leaders, researchers, and evaluators to share evaluation projects, findings and learning.  A rich and hearty exchange it was--too much to encapsulate here.  But here are a few morsels to whet your appetite for more detail in future reporting:

The Los Angeles County Arts Commission (LACAC) and the City of Los Angeles Dept. of Cultural Affairs (DCA) are trendsetting in areas of data collection and analysis. DCA, through Fellow Wendy Hsu, used rapid processing approaches and a discovery mindset to mine extant public art and grantmaking data from agency sources and Instagram “to raise further questions about what we don’t already know about the social and cultural reality of the communities that we serve” and implications regarding cultural equity and social outcomes. At LACAC, Bronwyn Mauldin showed how changing the sampling frame and way they ask questions to eliminate bias has dramatically changed the agency’s understanding of who is participating in Ford Theatre’s programs and is shifting conversations from community engagement to cultural equity. 

Kamella Tate, independent evaluator and former director of research and evaluation at The Music Center of Los Angeles County, called out thorny issues related to credibility such as evaluation scopes that are way too large to map all variables, and shortcomings of approaches such as appreciative inquiry that may leave out contrary cases or outliers. Courtney Malloy of Vital Research joined Tate in cautioning against the arts community’s tendency toward “exceptionalism,” i.e. the feeling that the arts cannot be methodically assessed for their impact.  Values-based approaches to evaluation can, in fact, gather quantitative data that is meaningful in terms of social justice, and qualitative data that honors the depth of the work and its transformational effects. 

The L.A. Evaluators Circle demonstrated the hunger for and value of peer exchange and building a cadre of connected professionals committed to the social impact of arts and culture and building a culture of evidence for the power of arts in social change work.  We are excited to facilitate more exchange in the future!

We invite feedback over the next few weeks on the two draft frameworks.  Review introductory documents and request review copies here:

• A Framework of Aesthetic Characteristics and questions to help assess aesthetic process and product/experience of art, culture and social justice work. Introduction (PDF)

To review, E-mail [email protected]   Subject: Aesthetics Framework

• A Framework of Values and related questions to help arts practitioners, evaluators, and funders plan and implement responsible and ethical evaluation and be intentional about evaluation practices that reflect values congruent with social justice. Introduction (PDF)

To review, E-mail [email protected]   Subject: Values Framework