Adapted from “Synthetic Credit Ratings and the Inefficiency of Agency Ratings,” by Doron Nissim from Columbia Business School.
Key Takeaways:
- Synthetic credit ratings, estimated using publicly available accounting and market-based information, can be a valuable alternative to agency ratings. They are nearly as informative and can sometimes predict changes in agency ratings months before they happen.
- The advantage of agency ratings over synthetic ratings is small — and even more so over the past decade, possibly due to SEC regulation after the 2008 financial crisis that limits rating agencies’ access to confidential information.
- Credit rating agencies risk losing value if they don’t improve their product over synthetic models.
Why the research was done: Credit ratings are a vital tool throughout the financial system, used for everything from monitoring credit and conducting lease negotiations to valuing fixed claims and estimating the cost of debt capital. But despite their ubiquitous value, not all companies are rated — and specifically, ratings are unavailable for the majority of public companies.
In this study, Doron Nissim, the Ernst & Young Professor of Accounting and Finance at Columbia Business School, develops and evaluates a pioneering model for generating “synthetic” credit ratings, using publicly available information. “Credit ratings are based, to a large extent, on accounting information,” Nissim says. “I was interested in whether I could generate synthetic ratings that would be close to what rating agencies basically offer when they do follow companies.”
His initial modeling showed that the synthetic ratings were viable. “I was surprised to see how good a relatively simple model was able to get compared to agency ratings,” Nissim says. “It led me to look into other aspects of what we can get from synthetic ratings. For example, do they provide incremental information about credit risk compared to agency ratings? And, if so, can synthetic ratings predict changes in agency ratings?
How it was done: Using a logistic regression model — which is typical of many credit rating models— Nissim used publicly available accounting and market-based determinants like firm profitability, leverage, volatility, and size to generate synthetic credit ratings. With this method, he estimated the model each month from December 1985 to December 2022 using all public US firms with available information on the credit rating determinants. He then evaluated the informativeness of the synthetic ratings.
What the researcher found: In his analysis, Nissim says he was surprised to find that agency credit ratings have only a small advantage over his synthetic model. He also discovered that this advantage has declined significantly in the past decade. He posits that the shift was possibly due to a 2010 Securities and Exchange Commission regulation that limited agencies’ access to confidential information. Nissim notes that “if the decline in the relative informativeness of agency ratings is indeed due to this regulatory change, it represents a significant unintended consequence of the regulation.”
Nissim also found that synthetic ratings can have a predictive value. They help explain differences across firms in credit default swap spreads, even after considering the information in agency ratings, and they predict changes in agency ratings in the following months. “These findings suggest that not only can synthetic ratings be used as a proxy for agency ratings when the latter are unavailable, but they can also supplement the information provided by agency ratings when evaluating credit risk,” Nissim says. “Moreover, there is some evidence that investors do not fully incorporate the synthetic ratings information when pricing credit risk.”
Synthetic models have been tested before, but Nissim’s model is unique in its emphasis on accounting considerations when incorporating variables into the formula. “These modeling improvements substantially increase the predictive ability of synthetic ratings, demonstrating the usefulness of a careful financial statements-based credit analysis,” Nissim says.
Why it matters: The strong performance of Nissim’s synthetic ratings suggests they can serve as an efficient substitute for agency ratings for the many companies for which agency ratings are unavailable. Nissim also documents declining incremental information in agency ratings compared to synthetic ratings, and he shows that his model provides information about credit quality that’s incremental to agency ratings, ultimately predicting movement in the agency’s ratings months before they change. These findings indicate that credit rating agencies have an imperative to significantly improve their offerings; if they fail to do so, they could find themselves rendered irrelevant.
Nissim concludes that “while some of the untimeliness of agency ratings may be due to the preference of credit rating agencies for rating stability, the substantial additional information provided by synthetic ratings suggests that rating agencies should be able to improve the overall quality of their ratings by incorporating the information conveyed by synthetic ratings.”
