“Measure Better,” an ImpactAlpha series in partnership with Acumen, features investment practitioners and thought leaders who are putting customers at the center of their impact strategies.
CDC Group, the U.K.’s development finance institution, wants to make sure measuring the impact of its investments goes hand-in-hand with delivering business insights to firms supplying products and services to underserved populations in developing markets.
Sara Taylor heads CDC’s Impact Fund, a fund of funds which has made about $175 million in commitments to twelve funds and other high-impact intermediaries investing to improve the lives of people across Africa and South Asia. Martina Castro, who joined CDC about year ago, designs and manages the impact measurement and management systems for the Impact Fund and the broader universe of intermediated investments at the $5 billion development finance institution.
Measuring the impact of impact investments – by talking to customers themselves
Managing a fund of funds, Taylor is keenly aware of the load impact measurement places on fund managers, and the companies the funds invest in. “The companies and fund managers we back are juggling a number of priorities as they execute often challenging strategies,” says Taylor. “We want to support our investees in working towards our common goal across the investment lifecycle.”
Castro agrees, and thinks impact investors need to shift how they go about impact measurement. The conversation, she says “is too centered around metrics and not enough about how we use these for impact management – that is, to minimize risks, manage underperformance and drive impact further.”
With Acumen’s Lean Data team, Taylor and Castro are undertaking a series of deep dives with some of the portfolio companies of the funds they’ve backed to turn a deeper understanding of impact into a driver of business value. The key: to have the fund manager, the company and the CDC team bought into the importance of the exercise. “The more this exercise taps into the companies’ pain points, the higher the likelihood that insights will be valuable and taken on board by the business.”
ImpactAlpha: For what types of investments do you measure impact?
Martina Castro: We measure impact across all our portfolio at CDC, which includes both funds as well as direct equity and debt. One of our intermediated investments is Energy Access Ventures Fund, a Nairobi-based fund investing in companies that provide energy access to underserved populations. Initially our approach to impact measurement was to regularly collect metrics that capture scale, or in other words, counting how many people were being reached with solar lamps and other products and services that companies were offering.
Impact investors can gain real feedback from their portfolio companies’ customers. Do they want it?
Where we are today, is trying to understand not only the scale of the impact that we’re having, but also who we are reaching and what type of impact they are experiencing. In the off-grid solar space, where there is now quite a lot of research available, we’ve been trying to complement existing literature and randomized control trials with specific insights that we’re generating with a whole range of tools, including Lean Data.
ImpactAlpha: It’s a step deeper than the number of people reached.
Martina Castro: Yes. Usually operational metrics are useful to understand scale: to know how many people you’re reaching. But you don’t necessarily know the demographics of who you’re reaching and what value they are deriving from the business. Traditional impact measurement also does not give you insights on how you can further drive the impact you are creating, for example addressing weaknesses in product design, or listening to what other needs customers have.
ImpactAlpha: Can you give me an example of how you’re using Lean Data?
Martina Castro: In one instance, we wanted to understand what demographics a particular solar pump company was reaching, and what type of positive outcome solar pumps have on people’s lives. We were trying to understand what were the value drivers to the customers. This type of insight cannot be readily captured with operational metrics and indicators that companies report against on a regular basis. This is where Lean Data comes in.
Sara Taylor: Lean Data is one of the tools we are using for our impact measurement and management system. We’re exploring others as well, as different tools can be better suited to different business model and types of impact that we want to measure and validate. The overarching principle that guides us is recognising that the companies and fund managers we back are juggling a number of priorities as they execute often challenging strategies. We want to support our investees in working towards our common goal across the investment lifecycle. This includes helping them in terms of impact management, making the impact insights as useful as possible to the company as well, as well as sharing learnings with the broader industry.
ImpactAlpha: To what extent then does Lean Data add value to the business?
Martina Castro: In our experience, this is directly proportional to how much the businesses are engaged with the exercise. The spirit in which we undertake our impact deep dive is one where we want everyone to be bought into the importance of the exercise and take something useful away from it. Even though we try to keep these studies lean, they are going to take up time, which is a scarce resource for growing companies. We found that the more this exercise taps into the companies’ pain points, the higher the likelihood that insights will be valuable and taken on board by the business.
ImpactAlpha: Can you give an example?
Martina Castro: To go back to the example of solar pumps, there the promoter was very engaged with the exercise. He genuinely wanted to understand what the problems of the prototype were. He had already thought of a number of adjustments he wanted to make for the product, but the Lean Data exercise helped him validate some of those hypotheses and helped him identify problems that he hadn’t expected. A better designed product drives value to the customer, and ultimately generates higher impact.
ImpactAlpha: What are the challenges of using Lean Data when your investment is going to a fund manager, not to the company directly?
Martina Castro: Whilst we understand there is a sound firewall between the investing arm of Acumen and the Lean Data team, there has been some scepticism on the part of the fund managers to have what is effectively a competitor, Acumen, come and undertake these exercises. That’s also why, amongst other reasons, we’re not just relying on Lean Data for our impact measurement and management. We expect this to improve now that Lean Data is being spun out of Acumen and forming an independent entity.
Customer survey data that makes it simpler to embrace the complexity of impact investing
ImpactAlpha: And opportunities?
Martina Castro: Lean Data has proven that it is possible to undertake impact measurement in a non-overly burdensome and business-friendly way. The team is also very quick at turning around projects, which is highly appreciated by the companies and the fund managers. The insights generated carry real impact and business value, and the Lean Data team has been excellent at marrying the interests of all parties involved. This means we are getting both impact measurement out of it, and actionable business insights on how to further deepen our impact and improve the financial bottom line.
ImpactAlpha: What other tools are you finding useful in deepening and adding nuance to your understanding of impact?
Martina Castro: There are a number of tools out there which we are using and testing. We are particularly focussed on an emerging set of tools which seek to simultaneously demonstrate impact and drive commercial performance. We are increasingly deploying these tools and will share lessons learned with a wider community of impact investors.
These tools include everything from geospatial analysis – using satellite mapping to gather and analyze large datasets in an increasingly cost-effective way – to behavioral analysis, which are suited to test and measure specific types of less tangible impact such as political and civic engagement.
Lean Data is particularly suited to gauge the direct impact created by customer-oriented businesses. We have also designed surveys with the Lean Data team that are focused on suppliers, and are working with them to create surveys that focus on measuring impact on employees. Where Lean Data is less suited is capturing other forms of impact, including impact that might be happening through a B2B relationship, on an ecosystem, a market, or on the environment. Lean Data also relies on self-reported impact, which can be subject to bias and doesn’t hold the sort of counterfactual and academic rigor that more experimental methods have. These are all aspects where Lean Data falls short. And that is okay for us. It’s just one tool out of many, and it’s adequate for a specific type of business model and impact.
ImpactAlpha: Where are the gaps in impact measurement practice?
Martina Castro: There is a big hype at the moment around impact measurement. The industry is moving from measuring operational metrics once a year to undertaking more in-depth type of measurements. Because traditional impact assessments such as RCTs are costly and complicated, people are starting to devise simpler tools such as Lean Data. There is much more that can be done in this space and we encourage new ideas on how to improve impact measurement.
What I find, however, is that at present the conversation is too centered around metrics and not enough about how we use these for impact management – that is, to minimize risks, manage underperformance and drive impact further.
ImpactAlpha: Would standardizing impact measurement bring institutional investors off the sidelines and into impact investing?
Martina Castro: The lack of clarity around impact measurement and management standards is potentially scaring some people off. There are a number of initiatives ongoing at the moment, and which we are supporting, to bring more clarity and consolidation to this space, such as IFC’s recently launched ‘Impact management principles’, the GIIN’s work on characteristics of impact investors, and the Impact Management Project’s dimensions of impact. Standardization of impact metrics will be key for further industry consolidation. But at the same time, standardization doesn’t mean that everything that is standardized needs to be collected. I encourage fund managers to carefully decide what aspects of impact they want to monitor and then choose the standardized metric that is closest to that.
Measuring and managing impact is not an easy endeavor, but investors can slowly but surely improve their systems and become more sophisticated over time. As LPs, we should not be rushing them through that process, and ensure that what we are asking of managers is fit for purpose.
ImpactAlpha: You’re saying do something, use the data you collect to improve the operations of the company and get better over time.
Martina Castro: Yes, try walking before running.
Sara Taylor: Before we get to measurement of impact and thinking about which metrics to collect, it is important that we establish upfront what impact we want to achieve, and assess how we think this investment is going to achieve this.
ImpactAlpha: A theory of change.
Sara Taylor: Yes, exactly. There are different terminologies which can be used, but, effectively, we want to be more thoughtful about how we think an investment will generate impact, and then test those assumptions throughout the life of this investment, both though regular data collection and through our deep dives.
The other consideration, is that impact doesn’t just happen directly at the company level. We are also looking at impact that we may be having at other levels, for example, at the overall market system level. As a fund of funds, we are also looking at the impact we want to have at the fund manager level: part of our mission is helping to create funds and build a capital ecosystem for impact investing and in frontier markets. So when we look at our own impact measurement, we don’t only limit ourselves to specific company metrics.
Martina Castro: Ultimately, we want to encourage investors to spend more time thinking about, “Why am I doing this investment? What am I going to achieve with it?” rather than immediately jumping to the impact measurement side of things. That is easily the moment when the most interesting conversations happen. Once we’ve understood our goals, then we can start thinking about what metrics make sense, how are we going to collect them and then test some of our impact assumptions over time using for example Lean Data.
