Why the Credit Rating Agencies Failed

I wrote this paper in April of 2009 for a class on national economic issues taught by the redoubtable Alice Rivlin. In it, I explored how the credit rating agencies contributed to the financial crisis, and suggested options for reform (some of which were incorporated in the Dodd-Frank Act).

The credit market crisis was a communal and colossal failure to evaluate risk.  Not only did we fail to recognize that the riskiness of some individuals could jeopardize the stability of the entire system, individuals failed to recognize that their own investments were in jeopardy.  From the irresponsible homebuyer to the financial wizard to the halls of Congress – no one was capable and willing to address the true risks.  The pre-collapse economy has recently been compared to a house of cards. The analogy is misleading, however, because the problem wasn’t with the building materials – it was with the foundation.  The entire system rested only on a few bad assumptions, many of which were made by the credit rating agencies.

There are currently ten Nationally Recognized Statistical Rating Organizations (NRSROs), the most major of which are Fitch, Moody’s, and Standard & Poor’s.[i]  The 2006 Credit Agency Reform Act required these credit rating agencies to submit to annual certification and to issue periodic financial reporting to the SEC,[i] but the law also prohibited the SEC from regulating the actual rating process itself.[ii]  By the time the NRSROs were registered in September 2007, they had already played their role in devaluing risk and spreading contagion; the damage was done.

The credit rating process for residential mortgage-backed securities and their accompanying collateral was flawed in many ways, which together greatly underpriced risk.[iii]  Such securities proliferated, beyond what was prudent for individual firms.  The regulatory structure itself was dependent on these ratings, as capital requirements are often pinned to the credit ratings of various assets.  Once the delinquency and foreclosures began in earnest, it became clear that the underlying risk assumptions had been false.  The rating agencies downgraded many of the troubled assets, and although they are clearly artificial instruments, this constricted the markets even further.[ii]

The Failures of Credit Rating

Lack of Competition: A frequently cited problem,[iv] and the target of many previous reform attempts, is the lack of competition in the credit rating industry.  The NRSROs might have needed to offer more prudent risk analysis if there had been greater competition.  As it is, the three major firms dominate the market.  Indeed, the Federal Reserve’s lending programs only accept collateral rated by these three firms.[v]  Unless underlying problems are addressed, however, increased competition might only produce more lenient ratings to attract customers.[vi]

Arbitrary and Opaque Procedures: A 2008 report by the staff of the SEC noted that “significant aspects of the ratings process were not always disclosed”[vii] – not to issuers, to regulators, nor even internally.  None of the three major NRSROs were found to have written procedures for rating residential mortgage-backed securities (RMBS) or collateralized debt obligations (CDOs),[viii] nor consistent methods of internal audits and error correction.  In many cases, when the SEC found deviations from standard rating procedure, there was no rationale recorded.  In light of these arbitrary methods, the new Chairman of the SEC, Mary Shapiro, highlighted the fact that customers may not have all of the information they need to make informed decisions at a recent discussion of the credit rating process.[iv]

No Due Diligence: A critical flaw in the process is that NRSROs were not required nor accustomed to performing due diligence on assets brought to them for assessment.[viii]  It was the expectation that the issuer of the security would verify the information about the loans underlying a RMBS, for example.  Although it was expected that anyone who uses the credit ratings would understand this, and perform their own due diligence, one wonders what the purpose of outsourcing the credit rating is in the first place.  Credit ratings are often made public, but private due diligence is not, and this renders a great deal of risk assessment uncertain to third parties.  Additionally, when there are automatic policy implications triggered by credit ratings, there is no regard for the reliability of the source information.

Insufficient Surveillance: It is customary for NRSROs to perform what is called surveillance, which is to monitor and update their ratings, but the process is not mandatory.  Indeed, the SEC staff report found that the surveillance procedures are less robust than the initial ratings process.  They note, “Performing adequate and timely surveillance is important, particularly when issuers of structured products do not make publicly available their due diligence information and underlying loan performance information, which would enable independent analysis by investors and third parties.”[ix]

Conflicts of Interest: The current pay structure has received a great deal of criticism.  It is the issuers of a security who pay for its rating, and credit rating agencies naturally have an incentive to please their customers.  There is an inherent conflict of interest.  New rules issued by the SEC attempt to prevent much of overt corruption, such as giving gifts over $25 in value,[i] but the ultimate regulation boils down to no more than an admonition not to do anything “which could conflict with providing ratings of integrity.”[ix]  It has been the intention of Chairman Shapiro to find “a way to realign incentives so that rating agencies view investors as the ultimate customer.”[iv]

Weak Economics: The SEC staff review found a disturbing weakness in the economic models used in credit rating.  For example, two of the agencies simulated macroeconomic variables randomly, and one used a static model of the economy.[x]  One agency used subprime mortgages interchangeably with standard thirty-year mortgages, ignoring their fundamental difference.  The raters also relied on the historical performance of subprime mortgages, despite the fact that recent years had been uncharacteristically strong, and few of the new loans had yet reset to higher interest levels.[x]  The model for CDOs was even more greatly oversimplified, relying on just five inputs, one of which was their own credit rating on the underlying asset.[x]  They professed predictions with certainty, based on earlier predictions, none of which were actually certain.

Insurmountable Complexity: At her recent discussion of credit ratings, Chairman Shapiro raised the possibility that some of these financial products are too complex to be evaluated with any degree of certainty.  To the extent this is true, ratings could be very misleading.

Possible Solutions

Historical Performance: A newly created SEC rule requires NRSROs to electronically publish a random sample of 10% of the ratings histories of issuer-paid credit ratings, in hopes that they will have more incentive to make accurate long term assessments.  Perhaps it would be better to take a look at the industry as a whole, and commission a high-profile report on the historical performance of each rating level over the last 5, 10, and 20 years.  Exactly how much value, on average, did a AAA bond lose since 2005, for example?  Such information will be useful as a cautionary tale to the current actors once we enter a recover period, as well as to history.

Invert the Pay Scheme: The conflicts of interest inherent in the current issuer-pay system are frequently acknowledged.  Many have suggested that investors should pay, or at least pay jointly with the issuers.  But, as once critic notes, “it is difficult to see how it would work in practice.” Ratings would have to be confidential to prevent investors from free-riding on each other.”[vi]  This helps explain why reforms to date have sought only to improve competition, and why some have called for the government to do their own ratings.[v]

Relative Ratings: Another proposal that could reduce the incentives for favorable ratings would be to grade on a curve, allowing each year only a certain percentage of all securities to be rated AAA by a firm.  This has been proposed by a British financial analyst named George Cooper.  It is a parsimonious solution, but it neglects the underlying problem that the credit rating process is faulty.  If nothing else is changed, it does not seem that the current system is capable of offering even merely a relative assessment.

Replace Grades with Robust Analysis: Simplicity should not be confused for transparency.  In this case, the oversimplification of a final stamp of approval rendered financial products too opaque to prudent risk assessment.  Although making accurate assumptions about future economic strength is often too difficult for even the combined power of the market, it was entrusted in a major way to analysts at just three firms.  The uncertainty of these assumptions, and the ability to subject them to sensitivity analysis, was hidden behind a stamp of approval.

Perhaps it would be better to abandon the grading process altogether.  Credit rating analysts could instead create a risk model for each type of security, sell it to an issuer or investor, and allow the many diverse users to supply the model with their own variables, with their own expectations of future behavior.  This would allow investors to choose exactly the amount of risk they would like, in a much more real way than the artificial tranches sorted by rating scores.  At the very least, this would prevent massive simultaneous downgrading of financial assets, as happened when the mammoth NRSROs changed their macroeconomic variables.  Altogether, it would offer the potential for a much more accurate and responsive corporate approach to risk.

Such a method for rating credit will be less prone to a false degree of confidence.  Any assertions made about riskiness can be accompanied with something akin to a confidence interval.  This means that CDOs that build upon RMBSs that are formed from individual loans will have less and less risk certainty, and cannot be misconstrued as a sure investment.  Financial products can safely be complex, but complex products cannot safely be oversimplified.  According to one SEC report, “The key issue here is not complexity per se but rather the extent to which complexity feeds on itself thereby helping to create or magnify contagion risk ‘hot spots’ that may have systematic implications.”[iii]

In many ways, this change would invert large parts of the current process.  Preliminary research and due diligence would be performed by the credit rating agency, and economic assumptions would be made by the investor.  The customer would not be buying a one-time product, but rather the service of a functional risk model, which would shift resources from the initial rating to surveillance.

If properly implemented, a constantly monitored risk model would be of value both to the issuer of a security, and to investors.  Both would be willing to pay for an individualized analysis of the many factors affecting the risk of a particular security, and this could alleviate the current conflicts of interest.

Of course, no economic model, no matter how rigorously stress tested, will perfectly evaluate risk.  However, allowing a more varied set of assumptions, as well as greater transparency in evaluating the risks, will greatly strengthen our capacity to prepare for future events.

Integration with Broader Regulatory Reform

In the current situation, a regulatory overhaul seems to be a forgone conclusion.  It is of course critical that we either restore the divisions of Glass-Steagall, allow regulators to oversee broader segments of the market, or do both.  Each financial actor will need to submit to direct, federal functional oversight that promotes transparency and enforces laws for investor protection.  The SEC, or a similar agency, will continue to strengthen the regulation of NRSROs in this manner.  Specifically, it would be wise to consider whether to grant the functional regulator the power to review and evaluate the credit rating process itself.

The reformed system will also need to develop a mechanism to evaluate and correct systemic risk.  The full authority and structure of this mechanism will be contested, but the information about the credit ratings systems will be critical for proper systemic risk evaluation.  There should be a federal regulator that assesses the robustness of the standard risk models, and anticipates situations where they will fail.

Strong, comprehensive legislation is necessary.  Currently, in part because risk is artificially evaluated, firms are rewarded in the short run by ignoring long run risk.  Additionally, the implicit government insurance on much of the financial industry promotes moral hazard – a discounting of the present value of future risk.  Lastly, the full cost of the risk undertaken by the financial industry is not internalized in the price of risk, because collapse of financial actors harms many others besides the financial traders.  Artificial incentives, moral hazard, and externalities combine to greatly under-price risk.  Reform of the credit rating system is a major piece of the solution.


The major regulatory response to the financial crisis was the Dodd-Frank Act, which included several credit rating reforms.  They did not go nearly as far as I’d suggested, but they did require disclosure of the credit rating agencies’ internal procedures and did empower the SEC to suspend or revoke credit rating licenses.  Perhaps most importantly, the reforms have removed all instances of official reliance on credit ratings.  If you’d like a good analysis, see this 2012 study from an economist at the New York Fed.

Photo credit: Esther Gibbons

[i] SEC Amendments to Rules for Nationally Recognized Statistical Rating Organizations, February 2, 2009.

[ii] SEC Staff’s Examinations of Select Credit Rating Agencies, July 2008. Page 4.

[iii] Counterparty Risk Management Policy Group III, Containing Systemic Risk, August 6, 2008.  Page 17-18.

[iv] SEC Chairman Mary Shapiro’s Statement at SEC Roundtable on Credit Rating Agencies, April 15, 2009.

[v] The Economist, The wages of sin, April 23, 2009.

[vi] Cooper, George., Rethinking Credit Ratings, March 19, 2009.

[vii] SEC Staff’s Examinations of Select Credit Rating Agencies, July 2008. Page 3.

[viii] SEC Staff’s Examinations of Select Credit Rating Agencies, July 2008. Page 18-19.

[ix]  SEC Staff’s Examinations of Select Credit Rating Agencies, July 2008. Page 23-25.

[x] SEC Staff’s Examinations of Select Credit Rating Agencies, July 2008. Page 34-38.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s