Demystifying Impact Part 4: Can Investors Stop Saying “Impact” in 2023?

Written By Heather McGee

The impact investing field faces a growing demand for post-investment impact measurement and management (IMM) as more investors demand evidence of their social impact. In response to this demand, our previous article outlined the main post-investment IMM frameworks in the field like the GIIN’s IRIS+ and B Lab’s B Impact Assessment. Each methodology has strengths and weaknesses, and firms might apply a specific method depending on their organization’s maturity, size, goals, and experience. 

While the impact investing field lacks an exhaustive ranking of IMM strategies, we can identify cross-cutting challenges among most IMM methodologies that can inform practitioners’ priorities. These challenges are often interrelated, spanning topics like poor data quality, arbitrary target-setting due to a lack of benchmarks, and the lack of standardized definitions and frameworks. The last challenge, which is the shortage of incentives to allocate resources to post-investment IMM, compounds the former issues by limiting the time and money organizations can devote to tackling them.

Challenge #1: We can’t even agree on the definition of “impact” 

Investment firms lack a standard, comprehensive definition for “impact.” Hagedorn and Doran of Goodwin Law (2017) offer different interpretations of the term “impact” in the field: 1) measuring impact at an individual investment level versus at a fund level and 2) interpreting impact as any investment that accepts below-market returns to achieve social impact versus impact as the result of any investment that grows companies and thus (in theory) creates more jobs. 

The second example is especially concerning, as it presents conflicting views of the definition of impact: if any investment can generate “impact” through the indirect creation of jobs, then below-market returns would not lead to a high value-add to society, as they would limit investors’ motivation and incentives to continue their financing. Conversely, if investors accept below-market returns in exchange for greater positive social change, then it is this acceptance of lower financial return that can catalyze more “impact.” 

Furthermore, ESG-based investments present even more divergent conceptions of impact, aggravated by ESG rating agencies’ lack of methodology transparency and conflation of “risk” with “impact.” Other interpretations of the term might be related to the primary methodology used to measure it. Organizations using a theory of change for their investments understand ”impact” as the last step in their logic model, referring to the overall, broad effect of its investment that results from short-term outputs and medium-term outcomes. For other organizations, these short-term outputs and outcomes might constitute their interpretations of “impact.”

While the general public correctly interprets the term as any positive non-financial change, the investment field must agree on a consistent definition or at least resolve and contextualize conflicting definitions of “impact.” The Impact Management Project’s 5-dimensional impact framework offers a useful starting point for investors and practitioners who seek to abide by industry-compatible definitions while maintaining enough open-mindedness to account for each organization’s specific context. 

Challenge #2: Bad data leads to questionable results

Impact results are only as good as the data used to determine them and, as previously discussed, as the underlying definition of “impact” itself. As highlighted in Demystifying Impact Part 1, the GIIN’s 2020 report on “the state of IMM” cites data transparency, availability, and quality as some of the biggest challenges to overcome in advancing IMM. For post-investment IMM, data transparency is crucial, as it allows investors to understand what over- and underperformance on social impact truly mean. 

The Wharton Social Impact Initiative (2022) describes a common and alarming practice: impact investing investors tend to report metrics to market their success rather than truthfully evaluate and disclose their impact. To illustrate, the GIIN’s latest annual impact investor survey from 2020 showed that an overwhelming majority of respondents believed they met or outperformed their social impact expectations (99%). Given the complexity of achieving social impact, especially through indirect approaches like impact investing, firms’ (self-reported) high performance might indicate an issue with the data used for target-setting rather than prodigious success in achieving positive social change. Without knowing their peers’ impact performance, organizations have no option but to rely on their own set targets to define “success.”

How Psychological Biases Influence Investment Evaluation

Behavioral research elucidates biases, fallacies, and pitfalls that investors can fall prey to throughout their investment management. IMM processes can be self-serving in that investors likely use sustainability or impact reporting to attract new LP capital and attractive startups to their table. Therefore, even well-meaning investors are incentivized to make their IMM reports as attractive as possible. But investors should be aware of some of the most important biases that can knowingly (or unknowingly) color their judgment, such as 

  1. Self-attribution bias: Investors overestimate or “over-weight” the influence of their capital, network, and advice on their portfolio companies. 

  2. Confirmation bias: Investors selectively filter certain language and data points that a company provides that fit with what investors assume the company’s impact is.

  3. Regret aversion bias: Investors might communicate to their LPs and stakeholders that their portfolio company had an impact to justify the resources investors allocated to said company throughout the diligence and investment process. 

Self-attribution bias, in particular, can be difficult to resist when self-reporting. Moreover, correlation and causation are easy to conflate in the absence of a research background. Investors should work with third-party research partners to confirm the extent to which their investment contributes to a positive change. 17AM is building an ecosystem of anchor institutions, scientists, and industry experts to better understand its individual contribution to place-based change. We encourage our stakeholders to engage with us to assess their contribution to sector-based change.

The second biggest challenge identified in the GIIN’s IMM 2020 report was the inability to compare impact with market performance, attributed to a lack of benchmarks and transparency in the field. It’s worth discussing, however, the contextual propriety of creating uniform, standardized metrics to assess social impact. Smaller organizations might prefer methodologies and metrics tailored to their context, while larger organizations might benefit from standardized approaches that they help build.

Challenge #3: Social change is complex and interdependent

The long-term, interdependent, and context-dependent nature of social change transcends the corporate world’s understanding of linear, short-term financial change. Thus, most impact measurement efforts tend to focus on activities and outputs (e.g. number of people trained) as opposed to systemic, long-term outcomes (e.g. income gains over 5 years), as revealed by E.T. Jackson and Associates Ltd.’s 2012 review of impact investing. 

Furthermore, the scale and time horizon of social impact depends on the type of intervention and issue the investment is targeting. Due to this context-dependency, some investments might appear more successful when other exogenous, enabling factors are in place. Observable changes in outcomes are not tantamount to attributable changes due to the complexity of isolating outside factors (Reeder and Colantonio, 2013). The concept of additionality is paramount in this analysis: would the outcome have occurred without the investment or the firm’s actions? This assessment is technically difficult, but advancements like those of J-PAL (Abdul Latif Jameel Poverty Action Lab) at MIT in randomized controlled trials (RCTs) and meta-analyses collect data from several sources and could potentially add statistical rigor to the impact investing field. These more sophisticated impact measurement practices, however, are resource-intensive and remain impractical for most organizations. 

When it comes to social change that is intrinsically systemic (e.g. poverty eradication), there might be a fundamental difficulty in the ability to measure impact: real impact is rarely achieved by a single organization, but rather by the combination of interactions by different actors striving for a common goal (Ebrahim and Rangan, 2010). This is why collaboration and design thinking are essential, not only among impact investing organizations in the financial sector, but also with cross-sector players in the government, non-profits, and NGOs, specifically those in development, monitoring, and evaluation.

Challenge #4: Incentives for investment in IMM are scarce

While impact measurement and management (IMM) is generally subsidized by governments or philanthropies in public and civic sectors, private impact investing firms bear the cost of IMM, which leads to a higher overhead cost. Without strong internal incentives, firms’ allocation of resources to IMM might be too lean, leading to more superficial measures of output data as opposed to systematic investigations of impact. 

The lack of incentives might be due to a variety of factors, such as survey fatigue of impact investing beneficiaries, few formal incentive structures for practitioners in delivering social impact, and a perception of low value for robust measurement. The last factor appears to be at odds with the increased appetite for data-backed evidence of social returns: if more investors are demanding proof, then why would the value of IMM be perceived as low? 

The key to understanding this discrepancy lies in the word “robust.” The desired level of evidence varies among different kinds of investors depending on their commitment, background, and motivation. Some might seek a general, broad perspective on impact while others demand more concrete, rigorous proof. As long as the former type of investor makes up the majority of impact demand, the case for resource (time and money) allocation toward IMM will continue to lag. 

Closing Thoughts

Impact investing faces pivotal challenges in concretizing  the definition of “impact.” A more deliberate approach to measuring and managing impact is required instead of the passive, symbolic gestures that characterize ESG investing, the most popular form of impact investing. 

We are starting to observe optimism in the IMM space as networks of impact investors like Toniic engage in data and evidence-sharing across portfolios and organizations like the GIIN’s IRIS+ and the United Nations’ Principles for Responsible Investment (U.N. PRI) lead the charge in improving their IMM standards, practices, and collaboration. 

COP27’s recent emphasis on the trillion-dollar annual investments needed to meet the goals of the Paris Agreement is a rallying wake-up call for all sectors, including the capital markets. 

To help meet the global financial needs to combat climate change and inequality, impact investing practitioners must be authentic, ambitious, and transparent in the impact we claim to catalyze.

Heather McGee
Previous

Demystifying Impact Part 3: The Best IMM is Scientific and Intentional