Marc Maxson

Think about the most fun you’ve had doing charity work. What was it that really appealed to you? Was it the smiling faces of kids playing a sport or painting a mural? Maybe it was the moment you realized someone’s life would be forever changed by the small token of love that a program enabled one person to give another.

Do you know what those moments have in common?

First, they are significant on an emotional, social, or metaphyiscal level—and so no traditional evaluation is well-suited to quantify them.

Second, these moments belong to those whose lives have changed. Your impact, as the person who helped make it happen, should not be the focus (unless you enjoy being self-centered and alone in the world).

So why do we continue to act as if “quantitative” surveys about our own “impact” are smart?

My decade working as a neuroscientist with actual “quantitative” data enables me to confidently dispel this notion once and for all. Here me out:

• Social change is social. That means it depends on people. Lots of them. People lie, especially in surveys, and often with the best intentions. Self-reports from people are not quantitative.
• So that’s why we have statistics, right? Inferential statistics depend on random sampling, and sampling is almost never random given the reasonable time and cost constraints placed on nonprofits.
• Even more alarming, statistics has no really solid way of telling if the sampling was done randomly.
• If random sampling is a problem, then results will not be reproducible over time and in different places. That’s why a lot of high-paid people interpret them and argue over methodology. But I think that’s a distraction from the core problem—which is our obsession with extrapolating from brief and tiny samples of life to broad and timeless descriptions of social change and impact.
• If you want quantitative data about people and social change, it’s probably more practical to transform our evaluation tools into a regular part of daily life—like Facebook or Google—so that we’re constantly looking at tens of thousands of bits of knowledge instead of just a few hundred.
  

That is why GlobalGiving has been running a massive experiment around storytelling in East Africa since 2010. A standard evaluation would never work for us. It would be both too costly and too limited to cover our 530 partner organizations in Kenya and Uganda who typically receive about $6500 in a year through our online fundraising website.

So how did we merge our “evaluation” with life’s activities?

We worked with our partners to find and train thousands of young people. We targeted the poor, typically girls—the kinds of people denied opportunities in their community—and gave them a job as a scribe.

For 15 cents a story, they would go out and interview people each month and collect two stories from each about two different “community efforts.” Anything that a person or organization tried to do to help the community was a valid story.

Since this was an open-ended, continuous, and broad survey, we learned a lot more about every kind of social problem. Certain themes emerged again and again, such as child abuse, problems paying school fees, and runaway street children. And with 40,000 stories and counting, when a story theme emerges, it really does—in the sense that 25 stories are anecdotes, but 2,500 stories are data.

This is not a random sample, but we could easily collect a million stories by 2016. Because we are not pushing people away from their social circles and into some artificially controlled “random sampling” environment, we get more stories more often. This allows us to aim for a large and diverse sample instead of a random one.

In the long run, we will be able to do what standard evaluations cannot—start to see if trends around, say, child abuse, rape, and hungry homes are the same over time and across many locations. But even more important, tens of thousands of Kenyans and Ugands are finally getting to have their say.

These stories are all online, and GlobalGiving is only the steward; the stories belong to the storytellers and we work to ensure their voices are heard by all the organizations working in their community.

This has been difficult, requiring us to develop new tools for visualization, real-time SMS feedback & polling, and define a strategy for hosting regular community meetings across the region.

Being a steward of stories, and a servant to the community, is the smart way to view our place in the world. Instead of asking, “are we making an impact?” we should have been asking, “are things changing for the better?”

Were the goals of the community attained?” I’m convinced that defining “impact” and our own attribution to it is a misguided dream spawned from the minds of the powerful and self-centered. And I am not alone.

The next time a funder asks you for proof that your work was the reason the goal was achieved, ask them for proof that the dollars they gave you were the crucial input in you achieving your goal. Sounds absurd when you flip it around, doesn’t it?

The world is full of storytellers. With more vision, we should be able to build the means to capture life in greater depth, so that we will know if things are improving. If we’re doing our best, things will, as singer Mark Erelli explains:

I wonder sometimes what I will pass on

And how much can one voice do with just a song

Sometimes injustice and indifference

Are the only things I see.

But I refuse to let my hope

Become the latest casualty.

So I'll sing of love and truth

And try to practice all I preach

And if I can't change the world

I'll change the world within my reach.

If we show them all the tools

They might leave this place in

A little better shape than me and you

After all, we are only passing through.