The volume of sustainability data is growing fast. Greenhouse gas emissions, for example, are now reported or estimated for almost all public companies. This is great news for investors who want to allocate to companies based on their carbon footprints.

Beyond emissions data, the increase in broader company sustainability information can present meaningful challenges. For example, corporate sustainability reports may run a hundred pages long, differ substantially from one company to the next, and may not contain all the information that interests investors. As a result, it is only natural for investors to look for help in deciphering environmental, social, and governance (ESG) data.

The ESG Ratings Shortcut

At first, ESG ratings may appear a promising tool to navigate this complexity. Providers of ESG ratings may look at hundreds of reported and estimated variables for a single company and boil them down into a single ESG rating. Individual company ratings may then be aggregated into fund and index ratings or scores. Thanks to their convenience, ESG ratings have grown in popularity in recent years.

Globally, regulators have expressed caution. For example, in 2020, then-SEC Chairman Jay Clayton stated that he has “not seen circumstances where combining an analysis of E, S, and G together, across a broad range of companies, for example with a ‘rating’ or ‘score,’ particularly a single rating or score, would facilitate meaningful investment analysis that was not significantly over-inclusive and imprecise.”

Inherent Subjectivity

Beauty is in the eye of the beholder. Often, so too is sustainability. ESG ratings providers frequently disagree on company ratings. The correlation between the ESG scores of different ESG ratings providers has been estimated at 0.54, and even lower when looking at the individual E, S, and G pillars. It is common for a company to be identified as best-in-class by one provider and as just average by another provider. In comparison, the correlation in the credit ratings assigned by Moody’s and S&P is 0.99. These findings are consistent with Dimson, Marsh, and Staunton (2020). Exhibit 1, reproduced from Boffo and Patalano (2020), shows some examples of companies with a high level of disagreement.

Exhibit 1

Source: Boffo, R., and R. Patalano (2020), “ESG Investing: Practices, Progress and Challenges”, OECD Paris, https://www.oecd.org/finance/ESG-Investing-Practices-Progress-Challenges.pdf Note: Sample of public companies selected by largest market capitalization as to represent different industries in the United States. The ESG ratings are transformed using a projection to the scale from 0 to 100, where 0 represents the lowest rating and 100 the highest rating. The issuer credit ratings are transformed using a projection to the scale from 0 to 20, where 0 represents the lowest rating (C/D) and 20 the highest rating (Aaa/AAA). Sources: Refinitiv, Bloomberg, MSCI, Yahoo finance, Moody’s, Fitch, S&P, and OECD calculations.

Disclosure: Values range between -1 (strong negative relationship) and +1 (strong positive relationship). Values at or close to zero imply a weak or no linear relationship. Correlation coefficient values less than +0.8 or greater than -0.8 are not considered significant.

The sources of dispersion across ESG ratings include differences in what is measured, how it is measured, and what weight is assigned to each variable. Because ESG ratings often look at dozens of variables, and because detailed methodologies and score attributions are generally not publicly available, understanding where discrepancies come from can be challenging.

This complexity means that ESG ratings may not be effective at achieving, and sometimes even work against, the sustainability objectives of investors. A recent OECD study found a low correlation between the ESG scores and the E pillar scores of three rating providers. Even less intuitively, it also found a positive correlation between the E pillar scores and carbon emissions for two out of the three providers. In other words, a strong environmental rating was associated with emitting more, not less. Dimensional’s research has also uncovered a statistic that may give environmentally minded investors pause. A key finding in our 2021 study notes that “sustainability” funds vary greatly in reducing exposure to greenhouse gas (GHG) emissions, as one in four US-domiciled “sustainability” branded funds reduce greenhouse gas emissions exposure by only 11% or less compared to the Russell 3000’s emissions exposure.

Overall, given the subjectivity inherent in ESG ratings, we believe they should be viewed not as objective ratings, but as opinions—not unlike the buy/hold/sell opinions that have been issued by sell-side analysts for decades. When using ESG ratings from one provider to allocate assets, investors should be aware that other ratings providers may have dramatically different opinions and ratings.

For example, indices based on ESG ratings may deviate significantly from one another. As of the end of March 2021, the MSCI World SRI Index,¹ which emphasizes companies with strong ESG ratings, assigned a weight of more than 12% to Microsoft—nearly four times its market capitalization weight—but excluded Apple and Alphabet. By contrast, in the FTSE Developed ESG Index,² all three companies were overweighted. It is important that investors understand how these investment decisions are arrived at—but the opacity, complexity, and subjectivity of ESG ratings methodologies may make this understanding difficult to achieve.

From ESG Data to Robust Strategies

Rather than using generic ESG ratings, investors should first identify which specific ESG considerations are most important to them, and then choose an investment strategy accordingly. An example may be reducing exposure to companies with high emissions intensity. The broader the set of objectives, the more difficult it can be to manage the interactions among them. A “kitchen sink” approach that integrates dozens of variables may make it hard for investors to understand a portfolio’s allocations and may lead to unintended outcomes.

Investors should then take a close look at the data. Not all ESG data are created equal, and certain disclosures are more reliable and robust than others. For instance, data that rely on voluntary surveys may inadvertently favor large companies with well-staffed reporting teams, regardless of their actual ESG performance. Equally important is the ability of managers and fiduciaries to deliver meaningful sustainability reporting to their clients. Investment professionals should be confident that the ESG data they use can support the value proposition they seek to deliver to their clients.