Lean Data: The “how” of measuring impact at lower cost for higher value



Here’s an impact investing myth that needs busting: The proliferation of multiple frameworks for measuring social impact means that front-line employees, managers and investors are getting the kind of feedback and data they need to improve their operations — and deliver genuine impact.

With a tip of the hat to the Case Foundation’s Rehana Nathoo, who in her excellent opening to this Operation Impact series of essays called for continual myth busting, we’d like to bust “the metrics myth.” That’s what Jed Emerson, the originator of the term “blended value,” calls the misconception that the collection of proper impact metrics has become widespread. Sadly, it has not.

According to an annual survey of impact investors, unsophisticated measurement practice is one of the five biggest constraints to growing the impact investing market. With so many frameworks out there, why aren’t we making more progress?

We wrestled with this question as we began our own work creating the Lean Data approach to help build more impactful businesses by providing them with data on their social performance, customer feedback and behavior. Surely if we all agree it’s critical to measure impact, we’d be making greater progress? It’s not as if all that framework development is cost free. It consumes an awful lot of effort, for slim rewards.

To deliver the first, we’ve invested heavily in remote data collection. We’ve been leaders in experimenting with tools such as SMS, online surveys and simple phone calls for impact measurement. We have found they are highly effective at getting great data, especially in remote settings. And by focusing our data gathering and analysis on what investees want to learn, we’ve maximized the value generated. In practical terms this often means blending social performance measurement with other kinds of customer intelligence (e.g. Net Promoter Score, customer archetypes, customer care performance, and more).

This approach is not a free lunch. There’s the direct cost in terms of collecting data. But our bet is that once you start focusing efforts on data-collection rather than framework creation/metric-selection, you’ll find the value you create for each hour spent is dramatically improved. This bottom-up data approach also means that as funders or investors, we may have to accept that the metrics we want may not match those prioritized by front-line organizations.

Often, the fear is that this will prevent aggregation of impact data. Our experience is exactly the opposite. Companies in our portfolio have had surprisingly similar interests. Moreover, when they are made a data offer (“what data can we help you collect?”) as opposed to a data request (“you must collect this data for me”) our investees are much more excited about measurement.

Applying the principles of Lean Data, combined with a steely-eyed focus on getting end-consumer data at scale, we’ve transformed the measurement (and management!) of social impact at Acumen. In the past two-and-half years we’ve run more than one hundred Lean Data projects. In the process we’ve interviewed more than 35,000 investee-customers.

We’re still learning how to improve Lean Data in terms of both lowering costs and delivering ever greater value to investees. We’ve learnt one thing for certain: banishing the metrics myth is possible — if we re-orient our approach to measurement. It needn’t be so hard.

In Operation Impact, leading practitioners bust common impact investing myths with practical solutions. The series is brought to you by ImpactAlpha in partnership with the Case Foundation, Acumen, Bridges Fund Management, and ImpactSpace (ImpactAlpha’s open impact database). More from the series:

You might also like...