I asked an arts ed colleague the other day “What session would you like to see on a state-wide arts conference agenda to justify your time and travel expenses?” The answer: “How did you know your program made a difference?”

Wish: Granted. 

Logic Model

Before you even begin implementation, before you start writing the grant, before you talk to partners, make this your first step. Think your way through why you do what you do (long term impacts), what it’s possible to change (outcomes), what you can reasonably count without getting buried under useless data (outputs) and then construct what your program will actually do (activities). A good logic model is the path to knowing if you made a difference.

Evaluation Tools

Pre/Post surveys, focus groups, interviews, assessments: any of these (and more) evaluation tools can measure progress if they are implemented in alignment with the kind of program you are offering. Make sure you are following your logic model from activities to desired outcomes so you really measure what said you intended to change. Unintended outcomes are useful, but don’t discover them “on purpose” by using an inappropriate tool.

Butts in Seats (Quantitative Data)

It’s easy to dismiss statistics as less meaningful than qualitative changes. But if your goal is increasing access to services where there were none before, numbers can tell a powerful story. Providing dance residencies to every classroom at an elementary school that has no dance opportunities (something you will know because you used appropriate pre-service evaluation tools, see above), access to dance in that school has increased 100% because of you. The key is remembering to baseline before you start your work so you can measure how far you’ve come.

Site Visits

See the program in action. Notice the observable qualities that are shared between successful programs. Try to articulate what it is that “you just know” when something is working. Notice when students are engaged or have “ah-ha” on their faces. Document findings with photos, with blog entries, with facebook posts, in a notebook. Your observations are a valid measure.

Longitudinal Tracking

Whether it’s a one year, a three year, or a ten year program, re-connecting with participants after they leave is a powerful way to measure impact. Even if the final reports have already been submitted to the funders and your board has moved on to a new strategic plan, the satisfaction of returning to your logic model to check on the long term impacts will help inform future planning.

Stories

Perhaps the simplest, yet most unpredictable, way to know you’ve made a difference is to ask participants directly. Thank you notes after a field trip to the ballet, an end of the year conversation after pottery class, a presentation of flowers on opening night, a closing focus group session at the end of the orchestra season . . . A story of impact can be told anytime, anywhere. The question is: will you be prepared to capture it? Consider when it is appropriate to video record student interactions (even if the quality isn’t great, if you can capture the words, the story is yours). Keep your audio recording app handy on your phone and don’t be afraid to ask students to re-state what they just shared into the microphone—it might even be better the second time. Take time to photo-document letters, cards or other materials that express student views. Never go anywhere without your journal and a writing utensil because at the very least, you can make notes about what you heard in conversation.

What other methods have you used? Comment below to add your own suggestions.