Impact Myth: Some Impacts Simply Don’t Allow for Measurement
Pop quiz: What’s one thing that can’t be measured? Love? Happiness? Measuring love might seem crazy, but scientists have identified brain activity associated with love—and even how it predicts relationship outcomes down the line. As for happiness, the country of Bhutan uses a Gross National Happiness measure to assess progress that might not be captured in a GDP (as Center alumnus Alejandro Adler Braun is researching).
What’s all of this got to do with impact? In our ongoing series on impact, we’re addressing five myths and a question about impact definition. This week, we discuss our fourth myth: “There are some impacts that simply don’t allow for measurement.”
Like love and happiness, there are some things – tolerance, empowerment, social status or the social impact of a particular idea or person – that appear to be outside the realm of what’s measurable. But is that really the case? And what are the implications of saying that something can’t be measured?
Impossible, or Impractical?
When people say an impact can’t be measured, they often mean that they can’t practically measure that impact to the degree of accuracy they want and within the timeframe and budget they have. In other words, it’s not that the impact is technically impossible to measure, it’s that people believe it’s impractical to measure given known, existing tools and limited resources. It’s true that certain changes (e.g., an increase in income) may be much easier and cheaper to measure reliably than something like a change in social status of an oppressed segment of society (e.g., the Dalit population in India). However, saying something is impossible to measure makes it dangerously easy for donors and nonprofits to be let off the hook. What’s more, it can lead to a focus on what is easy to measure as opposed to what matters.
The Perfect Can Be the Enemy of the Good
There are genuine limitations on what most organizations can feasibly measure at any point in time. But even if an impact can’t be measured precisely, measurement is still possible. Perhaps there’s a proxy indicator—something easier to measure that is related to the goal you’re working towards. In the Dalit social status example, maybe publicly available data can give you a picture of changing employment opportunities available to this group. Or perhaps a limited survey of local perceptions of this group repeated over a period of time could give you a sense of whether attitudes toward Dalits are less negative, a proxy indicator for improvement in social status. And maybe you won’t be able to determine exactly what impact you had, but you can still get a sense of how much progress has been made overall in the areas you care about. The right approach will vary case by case, but nobody is off the hook when it comes to trying to measure their progress. Without some system of measuring, donors and nonprofits are flying blind.
Moving forward: Know your goals, know your trade-offs
Anything can be measured with enough time and money, and nobody can ignore their responsibility to measure their impact. But how do you combine those concepts to make decisions about your own measurement approach?
The key is in your goals. Good measurement clarifies your path to action. Put dollars towards the measurements that can help you or your grantees do their jobs better. You may not need to know your impact precisely, as long as you have reasonable evidence that you’re moving in the right direction. Or maybe you’re trying something brand new, and you need to measure many things very carefully in order to ensure that you’re doing more good than harm.
Measurement costs money, and there will be tradeoffs between the perfect, the good, and even the adequate. But as long as you’re making those tradeoffs with open eyes, you’ll be moving closer to achieving the impact you seek.