Blog Posts for social change

Let Evaluations Be Fun, Be Life

Posted by Marc Maxson, May 01, 2012 5 comments

Marc Maxson

Think about the most fun you’ve had doing charity work. What was it that really appealed to you? Was it the smiling faces of kids playing a sport or painting a mural? Maybe it was the moment you realized someone’s life would be forever changed by the small token of love that a program enabled one person to give another.

Do you know what those moments have in common?

First, they are significant on an emotional, social, or metaphyiscal level—and so no traditional evaluation is well-suited to quantify them.

Second, these moments belong to those whose lives have changed. Your impact, as the person who helped make it happen, should not be the focus (unless you enjoy being self-centered and alone in the world).

So why do we continue to act as if “quantitative” surveys about our own “impact” are smart?

My decade working as a neuroscientist with actual “quantitative” data enables me to confidently dispel this notion once and for all. Here me out:

  • Social change is social. That means it depends on people. Lots of them. People lie, especially in surveys, and often with the best intentions. Self-reports from people are not quantitative.
  • So that’s why we have statistics, right? Inferential statistics depend on random sampling, and sampling is almost never random given the reasonable time and cost constraints placed on nonprofits.
  • Even more alarming, statistics has no really solid way of telling if the sampling was done randomly.
  • If random sampling is a problem, then results will not be reproducible over time and in different places. That’s why a lot of high-paid people interpret them and argue over methodology. But I think that’s a distraction from the core problem—which is our obsession with extrapolating from brief and tiny samples of life to broad and timeless descriptions of social change and impact.
  • If you want quantitative data about people and social change, it’s probably more practical to transform our evaluation tools into a regular part of daily life—like Facebook or Google—so that we’re constantly looking at tens of thousands of bits of knowledge instead of just a few hundred.
    Read More

My Name is Rachel Grossman & I Am a Measurement Junkie

Posted by Rachel Grossman, May 01, 2012 5 comments

Rachel Grossman

I developed my deep fondness for assessment over 12 years in theatre education and community programming and I bring that affinity into my work as an artistic leader for dog & pony dc, the administrative leader for Washington Improv Theater, and a “chief experience officer” focused on community building and civic discourse through arts participation.

Why am I fond of measurement?

As a box-checker, it provides a tremendous sense of accomplishment. As a lifelong learner, it allows reflection on choices I make and their effect...in order to make stronger/more interesting or daring choices in the future. As a manager, it supports the creation and execution of successful programming and initiatives.

I grew up as an arts educator early in the assessment and evaluation movement in regional theatre education.

I learned some valuable lessons:

  • be realistic (you can only accomplish so much in 45 minutes with 30 third graders);
  • plans can be adjusted (and improved) when you know the endgame;
  • assessment is linked to impact and change;
  • if you can observe it, you can measure it.

It was no surprise when I fell head-over-heels for Theatre Bay Area and Wolf Brown’s Intrinsic Impact study, which reaches beyond measuring success by ticket revenue and surveys that only ask if audience liked/not a show.

Read More

Rethinking Social Impact: "We Can’t Talk About Social Well-Being Without the Arts & Culture"

Posted by Mark Stern, May 01, 2012 0 comments

Mark Stern

Mark Stern

Susan Seifert and I began the Social Impact of the Arts Project (SIAP) in 1994 in response to the attention that economic impact studies were gaining at the time.

We felt—in addition to their methodological flaws—that these studies captured only a fraction of the importance that the arts held for society. We committed ourselves to think through the theoretical and methodological issues involved in documenting the contribution that arts and cultural engagement have for community life.

Over the years, we’ve discovered many connections between the arts and social well-being, some of them quite surprising.

It turned out that the arts were associated with preserving ethnic and racial diversity in urban neighborhoods, lower rates of social distress, and reduced rates of ethnic and racial harassment. Perhaps most surprisingly, we found that the presence of cultural assets in urban neighborhoods was associated with economic improvements, including declines in poverty.

We used the concept of “natural” cultural districts to study neighborhoods where we found unplanned concentrations of arts organizations, cultural enterprises, artists, and cultural participants and documented that it was the social and civic engagement associated with the arts that seemed to drive these economic benefits and revitalization.

Over the past several years, we’ve been trying to re-conceptualize our findings and their meaning for the cultural community, urban public policy, and scholarship.

Read More

"You Can't Know What Will Grow from the Seeds You Plant."

Posted by John Bare, May 01, 2012 0 comments

John Bare

Let’s start with two assertions:

  • First, every meaningful social change movement for the last 1,000 years, at least, has been driven, in large or small part, by the arts.
  • Second, many arts-based civic works contribute little or nothing to individuals, communities, or societies.

It boils down to this: You can’t produce great social change without the arts. But there’s no guarantee that every arts-based program accomplishes something.

As with all interventions, whether arts or education or agriculture, much ends up on life’s cutting-room floor—or, if not tossed, left as a relic. If great art alone would suffice, Woody Guthrie’s Plane Wreck at Los Gatos would have changed the American experience for immigrant farm workers.

Let’s circle back to the first assertion.

  • Imagine what would have come of spiritual life in the last 2,000 years without the contribution of literature (pick a version of the Bible and say thanks to Gutenberg).
  • Imagine the LGBT movement without the contribution of theater (see Charlotte circa 1996).
  • Imagine the Civil Rights movement without Guy Carawan teaching We Shall Overcome to the Student Nonviolent Coordinating Committee at its founding in Raleigh in 1960.

Back in 2008, Guy’s wife, Candi, explained to me that Myles Horton had embedded music in every undertaking at the Highlander Folk School.

“Myles, before he founded Highlander, had been over to visit the Scandinavian folk schools. He had observed in Denmark that when people came together to work on problems, they did a lot of group singing. He kind of brought that idea back to Highlander. He was not a musician himself. But he was really supportive of anything that would help grassroots people feel stronger.”

Read More

Stories Have Impact, But How Do We Know?

Posted by Jen Gilomen, Apr 30, 2012 2 comments

Jen Gilomen

We’ve all had the experience of sitting in a dark theater and being moved by a compelling documentary story. And as documentary mediamakers, many of us have felt that power materialize during animated discussions that occur with and among audience members when the lights come up for the Q&A.

But how do we really know if our films are having an impact beyond the walls of the theater, and how do we know that our film is causing something besides “clicks” and “likes” online?

At Bay Area Video Coalition, we’ve come a long way in our understanding of impact evaluation and its purpose. It used to be that evaluation was another box to check off in order to satisfy the requirements of our funders. We collected surveys at the end of each training or program, and when funding allowed, we began to track our program participants and projects over a longer period of time.

Our thinking about the purpose of evaluation began to shift, however, when we received a multi-year grant from the National Science Foundation that included a funded, dedicated evaluator to help us design and implement an evaluation not just for reporting purposes, but to create feedback loops that would shape future programming throughout the program’s lifecycle.

Participating in the design of this evaluation freed us to shift our focus from one of conducting surveys and basic reporting (for others, usually as an afterthought) to one of viewing evaluation as an opportunity to better understand the real and long-term impact of our work—for ourselves, so we could become more effective.

Read More

Assessing What to Assess in Public Art

Posted by Jon Pounds, Apr 30, 2012 1 comment

Jon Pounds

I believe we need to be really careful about what results we claim public art produces. Inevitably, and understandably, we will be asked by someone to produce the evidence to back our claims.

Careless claims can be most difficult task prove and, unproven, confound the good efforts of us all.

My caution is not because I think public art does little; rather that some things we might believe (or hope) we do are difficult to prove.

There are recent examples of assessments of well-known cultural agencies that provided little or no support for the assumptions made about their work. Does that mean that the work is not valuable (or properly valued)…or that the assessment of its value is nearly impossible even when well financed and professionally investigated? Assessing public art is nothing like counting beans.

There are examples of attitudinal assessments that work for some cultural experiences—not so much public art.

If you assess attitudes before and after a theater performance, at the very least you are asking someone to reflect on an experience that is both visual and aural and one that they have invested some significant amount of time (and perhaps money) to experience. Similarly, if someone has gone to a museum, they have invested time (likely at least an hour) and money and have chosen the experience because they anticipate satisfaction of their desire. And, in both cases the producing agency can hope to see an increase in funding from annual memberships as a long-term form of assessment.

Can public art begin to match those conditions for assessment? No.

Read More

Pages