This article is part of “Investing in Racial Equity,” an ImpactAlpha series in partnership with Living Cities that explores how foundations and financial institutions can deploy capital to close racial wealth gaps and create opportunities in America’s cities.
Over the last five years, Living Cities, a collaboration of foundations and financial institutions working to close racial income and wealth gaps, has made investments in five Pay for Success projects. That’s fully one-quarter of the 20 such projects – sometimes called “social impact bonds” – that have been launched in the U.S.
In addition, Living Cities has made three Pay for Success Construction Loans and is an investor in the $10 million Pay for Success Fund managed by Reinvestment Fund. Living Cities’ portfolio includes a $1.33 million investment in the Massachusetts Juvenile Justice Pay for Success Project, $509,000 in the New York State Workforce Re-Entry Project, and $500,000 in the Salt Lake County REACH Project, all of which seek to reduce recidivism and increase employment, but for different populations and through different interventions.
Living Cities also invested $500,000 into the Denver Social Impact Bond to increase housing stability and decrease recidivism for chronically homeless individuals, and made a $650,000 investment in the Massachusetts Pathways Project to Economic Advancement, which aims to improve economic outcomes for immigrants and refugees.
As an early investor, Living Cities is clear to point out that Pay for Success is still in the early stages of the innovation curve and failure is part of the innovation process. To fail forward, we believe early adopters, like Living Cities, have a responsibility to be transparent about lessons learned.
For example, we’ve learned that there is a lot of potential for impact investors to partner with the public sector to take risks and prove out interventions that the government can scale. But it matters far more who we as investors partner with than that we are partnering.
It’s important to have people at the table who have budget authority to make decisions, political capital to get people on board, and operations experience to be able to successfully implement and oversee programs.
We’ve learned that the Four C’s of credit aren’t necessarily helpful in underwriting the risk of the investment, so we helped create the Four P’s of Pay for Success to help investors better understand and underwrite their risk.
And we’ve learned, as in our other impact investments, to be intentional in how we apply a racial-equity lens because many of the disparate outcomes we’re trying to address are rooted in structural racism.
Overall, we’ve learned that Pay for Success is not the end in and of itself—it’s a means to the end of getting better outcomes for people. It is too early to draw conclusive results about our Pay for Success investments, but not too early to draw lessons from early wins and challenges.
No. 1: Structure Pay for Success projects with the flexibility to respond to changing conditions.
We need to rethink how we evaluate outcomes and define “success” — and how those outcomes are tied to repayment to ensure all parties are equally sharing risk in a Pay for Success transaction.
Investor repayment in the U.S. has largely been tied to the evaluation of outcomes through randomized control trials, or RCTs, which are considered the gold standard of evaluation. RCTs have been integrated into Pay for Success on the premise that a service delivery model needs to be implemented to fidelity to ensure specific results can be achieved.
At the same time, a core element of Pay for Success is the flexibility to continuously improve and use data to inform decisions. Such a feedback loop allows the service provider to iterate its service delivery model to best serve its clients.
There’s an inherent tension between implementing a model to fidelity and being able to iterate the model in response to changing economic, political and social conditions.
The opioid crisis presents a clear example. Four years ago, the crisis wasn’t the epidemic it is today. Service providers are evolving their intervention models to incorporate substance abuse treatment to meet client needs, but that means the models being implemented today aren’t the ones proven to work four years ago.
The tension arises because the intervention of four years ago was used to determine thresholds for investor repayment. We don’t know whether the iterated intervention will be able to achieve the same levels of success. Service providers are doing what they should be doing — changing their models to best serve clients, but it significantly increases risk to investors.
Pay for Success projects can be structured to mitigate the rigidness of RCTs and more equally share risk among parties. The Massachusetts Pathways Project to Economic Advancement, for example, implements four separate, but related, program tracks. This allows for multiple pathways to investor repayment and flexibility for the service provider to iterate its model.
The Massachusetts project also appropriately matches the use of an RCT to what is needed. The first track expands the services of the service provider, JVS, and measures success via an RCT, because the Commonwealth of Massachusetts is interested in building a robust evidence base for those services. The other three tracks are based on work JVS has been doing already and repayment is not tied to an RCT. The structure requires JVS to perform as it historically has in order to repay most investors, which mitigates some of the expansion and scale risk associated with the first track.
Another example is the Salt Lake County REACH Project, which includes a secondary evaluation that tracks the impact of system-level changes. Significant policy events trigger a clause in the PFS contract that essentially allows repayment to investors based on a comparison of REACH project outcomes to historical rates, rather than on the randomized control trial.
We’ve also learned to avoid all-or-nothing deal structures. A couple of early projects linked outcomes for repayment purposes. That means investors only get repaid if all outcomes are achieved. This increases the risk to investors that no repayments will be made even if some positive outcomes have been achieved.
The timing of evaluations matters too. Pay for Success projects are playing the long-game. To truly know the effectiveness of an intervention in achieving, for example, a reduction in recidivism or sustained employment, you often have to wait three to five years. Interim evaluations on long-term outcomes should be used as a tool for learning, not linked to payments.
No. 2: Investing in outcomes for people requires a new way of underwriting.
When making our first PFS investments, it was hard to assess the likelihood of getting paid back. The “4 C’s” of credit – character, capacity, capital and conditions – did not really apply, and there was no fifth C – collateral.
So in partnership with Reinvestment Fund, Living Cities created a new framework to underwrite PFS transactions. We called the framework the 4 Ps of Pay for Success – policy, process, program and partnership. It includes all the elements we believe are crucial to the success of a PFS transaction. Read more about the framework.
No. 3: Engage budget officials along with high-level champions.
We’ve always believed that to maximize their full potential for enduring systems change, Pay for Success projects require the commitment of government champions at the highest level possible – someone at the level of governor, mayor or a county executive office.
We’ve learned that it’s also crucial to have someone with budget authority as well, such as a county’s chief financial officer, or someone from the State’s Office of Administration and Finance or the Comptroller’s office. These are the people who will ultimately ensure that funds are appropriated to make payments to investors if success is achieved. These officials figure out where funding streams for projects can come from. This is especially important if the goal is for government to scale the project in the future.
For example, in Salt Lake County, the County CFO, Darrin Casper, is an active participant on the Pay for Success project’s Executive Committee. He made the case to the County Council to set-aside the full amount of county success payments in escrow ahead of schedule. That eliminated the appropriations risk to investors.
On the flip side, we’ve been involved in a transaction that had public sector champions at all levels – the Mayor, Governor and even federal officials. Prior to closing, however, the State pulled the plug on the project. Officials chose to re-allocate the funds set aside for the Pay for Success project to close the previous year’s fiscal gap. We believe the key reason that happened was because no one from the Comptroller’s office, which was making the budget decisions, was actively involved in the project.
No. 4: Pay for Success projects won’t address racial barriers without intent.
Living Cities has evolved over the past five years to become more intentional about putting racial equity at the center of our work. Looking back, we should have been more deliberate about questioning whether projects create better outcomes specifically for people of color.
For example, at the time of investing in the Denver social impact bond, we required that the data in all outcomes reports be disaggregated by race so we could assess whether the program was successful for all participants, regardless of race or ethnicity.
However, as we’ve improved our racial equity competencies, we now know the importance of digging deeper to understand how projects intend to address systemic racial barriers. Are service providers aware of their own implicit biases? Are services provided with a trauma-informed lens? Are incentives aligned so that “success” cannot be claimed if only some participants improve?
To create systems change, we need to consider if a project is addressing root causes that are contributing to current disparate outcomes, many of which are rooted in structural racism.
No. 5: Pay for Success is a means to the end of getting better outcomes for people.
Five years ago, one of the reasons we were interested in Pay for Success was to learn if a market could be built that could influence government to invest in what works. Since then, we’ve seen deep systems change happen in places like Denver which, based on early promising results, has scaled the Denver social impact bond two years into the project.
But repayments to investors and systems-change are not happening consistently. We’ve learned that Pay for Success is not the end in and of itself. It’s a means to the end of getting better outcomes for people.
We’ve realized that Pay for Success is one part of a larger outcomes-based financing space. We should be taking the things we’ve learned and integrate them into everything we do to get to the broader systems-change of investing in results and outcomes.
Pay for Success today doesn’t look like it did five years ago. It will likely look different five years from now. That’s okay, as long as we continue to fail forward. Innovation requires iteration, if we’re to progress up the innovation curve.
Ellen Ward is the chief of staff and Sindhu Lakshmanan is a senior investment associate at Living Cities