Rachel Grossman

My Name is Rachel Grossman & I Am a Measurement Junkie

Posted by Rachel Grossman, May 01, 2012 5 comments


Rachel Grossman

Rachel Grossman

I developed my deep fondness for assessment over 12 years in theatre education and community programming and I bring that affinity into my work as an artistic leader for dog & pony dc, the administrative leader for Washington Improv Theater, and a “chief experience officer” focused on community building and civic discourse through arts participation.

Why am I fond of measurement?

As a box-checker, it provides a tremendous sense of accomplishment. As a lifelong learner, it allows reflection on choices I make and their effect...in order to make stronger/more interesting or daring choices in the future. As a manager, it supports the creation and execution of successful programming and initiatives.

I grew up as an arts educator early in the assessment and evaluation movement in regional theatre education.

I learned some valuable lessons:

  • be realistic (you can only accomplish so much in 45 minutes with 30 third graders);
  • plans can be adjusted (and improved) when you know the endgame;
  • assessment is linked to impact and change;
  • if you can observe it, you can measure it.

It was no surprise when I fell head-over-heels for Theatre Bay Area and Wolf Brown’s Intrinsic Impact study, which reaches beyond measuring success by ticket revenue and surveys that only ask if audience liked/not a show.

The Intrinsic Impact approach is comprehensive: ask artists what they thought the impact of their work would be on the audience and then ask the audience the same questions. Tabulate. Compare. Reflect. Discuss. Look at trends over time. Reflect. Learn. [insert sigh of contentment]

For me, the takeaway of the Intrinsic Impact work is that it positions artistic leaders, and eventually theatre artists, to be more intentional. They can base choices on the effect they want to have on an audience, and then to gauge that effect. It has the potential to result in increased engagement between artists/art/audience and to reframe the conversation about the role of theatre and value of art in civic life.

But here’s the thing: I’m feeling in the minority.

Back in the day, arts educators and teaching artists across the country resisted the use of outcomes-based assessment models with protests of:

Rachel teaching a class.

“You’re confining our creativity.”

“You’re devaluing our work.”

“You’re requiring us to justify our existence.”

And even: “You can’t measure what we teach.”

I believe this was largely a reaction to an unfamiliar practice first introduced by funders and not arts educators; this lack of familiarity exacerbated by the limited formal education training of most arts education and teaching artists.

I am seeing similar reactions to Intrinsic Impact work. A recent discussion started about asking audience to describe their experience of performances, but quickly morphed into objections to allowing the audience to dictate your next season. Wha?

I was reminded of objections from artistic colleagues not even a year prior:

“We can’t measure the effect of our work.”

“The audience will either ‘get it’ or not.”

“It is subjective—the audience makes their own meaning. There is no right or wrong reaction.”

“I know I’ve achieved it when I see it, but I can’t describe it.”

“The audience has no right to judge my art.”

Measurement is once more rejected as an unfamiliar practice from outside the field with no value—a perception reinforced because when we do measure success it is by butts in seats, dollars in hand, and favorable/unfavorable reviews.

The Intrinsic Impact approach is linked to the transformational power of live performance, not ticket revenue. As theatre artists, we all believe that artistic engagement has a positive impact: broadening perspective, expanding capacity for empathy, ability to deliberate, and desire to improve ourselves and our world.

I propose that art makers ease into measuring impact by trying some of the approaches of educators:

  • What is the essential question of the work--the open-ended question that frames audience inquiry?  What ideas or understandings will audience “uncover” through the performance?
  • What behaviors do you want to see the audience engage in during this performance? When you see them do those things, what message does that send you?
  • What activity do you want audience involved in immediately following a performance of this show?
  • What will the audience learn or be inclined to do?

Then:

1) Watch the audience during performances.

2) Ask the audience (comment card or emailed Google form)

  • What did the show make them think about?
  • Did they [insert list behaviors] during the show?
  • What questions about the work or for the artists linger after the show?

3) Look over all the answers.

This is your first step toward exploring the effect of your work.

Once you have waded into the pool, I promise: it gets much easier to digest the different areas or “constructs” of Intrinsic Impact. At first glance, it can be intimidating. But the only way we are going to take control of the conversation about “why art matters” and “art’s effect of the community” is to direct it and engage all the participants (e.g. artists and audience).

Measurement is scary. It requires us to make intentional choices. It holds us accountable to ourselves and the work we are dedicated to do first and foremost. But it shows the impact of our programming in action. It can tell the story of why we matter, and that’s a story we should all want to tell.

5 responses for My Name is Rachel Grossman & I Am a Measurement Junkie

Comments

May 03, 2012 at 2:41 pm

Pam: I had forgotten about the folder idea from Marty's blog! Shame on me for doing so too; but now that you reminded me, it seems like something I should adopt for the next show I am involved with. So easy to group and quantify these "anecdotes." Then the stories are transformed into "back-up" of data/metrics. I was also warmed by Katherine Gressel's first piece about establishing metrics and collecting reliable data. All these artists being creative about evaluation and measuring impact!

  • Please login to post comments.
May 02, 2012 at 10:26 pm

Rachel, thanks for proclaiming your passion for evaluation. Your blog and Marty Pottenger's (check it out!) underscore from an artist's perspective how crucial a role evaluation can play in understanding and improving creative practice as well as how that creative practice works toward other social or civic goals. Many artists work solo or with limited resources for evaluation, as Matt responded to your post. But your point about taking baby steps is well taken. I love the story that Marty tells about making file folders, each tabbed with a different indicator she is using to measure changes in police morale or public perception as part of the Art At Work project. Whenever she heard a story told related to these (and other indicators), she jotted them down and slipped them in the corresponding folder. Months or longer of doing this can lead to a body of evidence! It may not be the thousands of stories that Marc Maxson is seeing through GlobalGiving's scaled up work (check out his blog!), but it is a start. Thanks, too, for bringing forward WolfBrown's Intrinsic Impact study in relation to your interests.

  • Please login to post comments.
May 02, 2012 at 10:23 pm

Matt: I am interested in what constitutes a good or bad assessment for you when it comes to measuring the impact of a work of art? I have confidence that the quality of my assessments were sound (unsure if they were "good"). The fact that I was engaging in thoughtful reflection about the effect/affect of my work went over well with producers, funders, board members, etc. I conducted them via google form and paper/pencil, which kept costs low. In the end, if we don't take control of the measurement methods someone else always will be dictating what we should measure and what constitutes "success."

  • Please login to post comments.
Matt says
May 02, 2012 at 3:18 pm

Well, measurement seems to be working fine for general education. Right? Maybe not?

The problem remains: good assessments are costly and will not be funded. Bad assessments are cheap, and cheapen the process, and will be used. These will lead to standardization, which is highly problematic for the arts.

  • Please login to post comments.
Christina says
June 07, 2012 at 4:21 pm

Thank you for this post, Rachel. I have been working in the arts for a decade and have always been interested in evaluation and measurement. What is often surprising to me is that program staff who object to "intrinsic impact"-style evaluation tend put a lot of stock in metrics such as tickets sales and attendance. Although everyone feels good when a program is sold-out, the sold-out show does not demonstrate impact. Adding another dimension to performance measurement beyond pennies & people, which are often subject to external forces as well as limits in capacity, should be welcome in so much that it acknowledges the magic of art. Looking forward to hearing more from you.

  • Please login to post comments.