Using Technology in CECL Model Validation

By Kevin J. Brand, CFA, CPA; Michael J. Budinger, CAMS; and Ryan A. Michalik, CFA
11/14/2018
Using Technology in CECL Model Validation
As the effective date for adopting the current expected credit loss (CECL) methodology for estimating loan losses approaches, banks, credit unions, and other lenders must gear up for significant changes in their accounting and financial reporting practices. One of the most challenging of these changes involves the question of how to appropriately validate the models lenders use to estimate expected future losses, a process that is inherently complex and data-intensive.

Fortunately, new technology solutions are available to make model validation more efficient while delivering enhanced analytical insight. Banks and other lending organizations should familiarize themselves with regulator expectations regarding model validation, and understand the critical capabilities to seek when selecting credit risk management and model validation solutions.

Model risk in the CECL environment

Because the new CECL methodology for estimating credit losses differs significantly from the previous incurred loss method, banks and other financial services organizations will need to make significant process changes, implement new technology platforms and financial models, and adapt their existing technology, data infrastructure, and governance structures.

The new CECL standard eliminates the “probable” threshold for loss recognition. Instead, lending organizations will be required to estimate reserves using a more forward-looking approach that takes into account expected losses over the expected life of the loan.

Historical loss information still is the foundation, but this information will extend over a longer period and need to be adjusted to fit current conditions, including asset-specific risk characteristics. CECL also allows for adjustments that reflect reasonable and supportable forecasts of future economic events and their impact on the portfolio. Lenders also should factor in expected prepayments when estimating the expected life of a financial asset.

The goal is to accurately represent the expected credit loss over the remaining contractual term of a financial asset or group of assets. To do this, lending organizations will need to deploy new or revised modeling approaches that are best suited to the risk characteristics of each portfolio segment. 

There are a number of considerations that can lead to a CECL model being identified as high risk. One issue is the direct and material impact these models have on an organization’s financial statements, especially on earnings and capital. A closely related issue is the reputational risk that would be associated with any restatement that might be needed because of a model error.

CECL models also are likely to have an influence on loan origination decisions, including pricing and new product offerings. Moreover, while they are relatively new and immature, CECL models also are inherently more complex than the previous methodologies, both in terms of their data requirements and the computational and programming processes that are involved. For these reasons, CECL models also are likely to draw close scrutiny from regulators and auditors, which introduces yet another area of risk.

Ideally, technology that supports alternative views and alternative scenarios will be incorporated as part of a broader loan portfolio risk management solution. Such a solution would enable lending organizations to conduct technology-enabled model testing and validation that is transparent and efficient, while providing comprehensive analysis of CECL reserves and portfolio risk.

With the implementation dates of the new CECL standard now at hand, lenders must move expeditiously to develop, implement, and validate CECL methodologies prior to implementation. For public business entities that are U.S. Securities and Exchange Commission filers, the new standard goes into effect for fiscal years beginning after Dec. 15, 2019, including interim periods within those fiscal years. For all other public business entities, the method goes into effect for fiscal years beginning after Dec. 15, 2020, including interim periods within those fiscal years.

Regulators’ expectations regarding model risk management

As banks and other financial services organizations expanded their use of various financial and economic models over the years, regulatory agencies began issuing guidance regarding their expectations for managing the risks inherent in such models.

The most comprehensive and widely cited guidance was issued by the Office of the Comptroller of the Currency (OCC),  and the Federal Reserve Board in April 2011, and was adopted by the Federal Deposit Insurance Corporation in June 2017. This joint publication, Supervisory Guidance on Model Risk Management (OCC 2011-12, SR 11-7, FIL-22-2017), spells out regulators’ expectations across a broad range of model risk management issues.

A critical component of the guidance relates to the need to validate models appropriately. Broadly speaking, regulators’ expectations regarding model validation encompass six general areas:
  1. Model validation must be independent from the model development and related challenge initiatives.
  2. Those carrying out the validation effort must possess appropriate levels of technical, business, and statistical expertise.
  3. In addition to confirming that the model is operating as designed, the validation process also should identify model weaknesses and limitations.
  4. The validation effort should encompass vendor models as well as those developed internally.
  5. Senior bank management should provide appropriate oversight of the model validation program.
  6. Validation activities should be aligned with the specific risks of the model in question.
In addressing these criteria, lenders should apply a structured approach that demonstrates to examiners that they have a clear understanding of the overall validation process as well as the particular risks associated with the CECL-related models.

Five major components of CECL model validation

At a high level, the use of models encompasses five broad areas of risk. Each of these components requires testing and effective challenge in order to provide reasonable belief that a model is operating as designed and intended. The five components are:
  1. Design and development. The model validation effort evaluates the intended purpose of the model, the model logic and functionality, its alignment to the purpose, the assumptions and limitations of the model, and the methodology used to develop it.
  2. Input processing. The data inputs relied upon by the model must be assessed for accuracy, reliability, and completeness. The use of appropriate data proxies and third-party data also must be validated, along with the integrity of the data transfer from source to model.
  3. Implementation. The processes used to implement the model must be examined and tested, along with the related model configurations and settings. This effort also involves assessing how the design and functionality of the model are integrated into the organization’s business setting.
  4. Output and use. This component of the validation effort encompasses various forms of outcome analysis, including back-testing, sensitivity testing, and benchmarking to assess the model’s function and output.
  5. Performance. The final component validates the established plan to assess the performance of the model on an ongoing basis. The comprehensiveness and clarity of model output reporting also is assessed.

Technology solutions – what to look for

Fortunately, technology now is available to help lenders execute the complex model validation process that CECL implementation requires. Packaging a technology-enabled CECL model validation solution, industry expertise, and meaningful portfolio insights will position lending organizations for success throughout CECL implementation and beyond. 

Establishing a model validation plan and identifying a model validation partner for CECL can be a challenge in itself. In addition to applying sound vendor qualification and selection processes, lenders should make it a point to look for certain critical features and capabilities related to the CECL methodology and the associated model validation processes.

Many of these features relate directly to the five model validation components discussed above. For example, in terms of basic design and development, the solution should apply modeling methodologies and settings that replicate the organization’s CECL model, but also be capable of challenging the methodology by running alternative approaches and comparing the results. 

Accurate and comprehensive risk identification have always been important in calculating reserves. The adoption of the CECL standard means a more defined risk identification process is now in order. In the same way, the solution also should develop portfolio segments that mirror the segmentation used by the lender. It also should be capable of comparing outcomes using alternative segmentations with ease, in order to challenge and identify potential management bias as part of the validation process. 

In addition, it should offer built-in portfolio analytics capabilities that allow validation of the segmentation methods used to identify and capture all relevant risk characteristics, such as loan purpose, vintage, term, credit score, industry, collateral, size, geography, and other critical risk drivers. The portfolio analytics also can provide the lender with insights into their portfolio to help identify risks in the portfolio, monitor concentrations, identify loan pricing discrepancies, identify data concerns, and improve overall internal credit analysis. 

In terms of input processing, any portfolio risk management solution that is to be used for CECL model validation should use a secure portal to load and store source data. It should have a capability of mapping the model input data and performing required transformation of that data into the modeling system with a traceable process flow. With the input data, the solution should have mechanisms to identify any data quality concerns or anomalies that could affect the models, reconcile the loaded data to validate the model inputs, and ensure that the data the model relies on is accurate, reliable, and complete.

Finally, any technology solution that is chosen also should address various performance and output validation needs, including the ability to perform sensitivity testing, benchmarking, and back-testing. These capabilities are essential to model validation in order to compare the results of various scenarios and settings with the organization’s calculated CECL reserves.

With implementation dates for CECL fast approaching, now is the time for financial services organizations to focus on selecting a model validator with a technology-enabled solution that can meet regulatory expectations and support the more complex modeling and validation efforts that will be required. The right choice, carefully implemented, can help make the transition successful while also providing lenders with powerful new risk and portfolio analysis tools.
 

Contact us

Kevin Brand
Kevin Brand
Michael Budinger - Large
Michael Budinger
Principal
Ryan Michalik
Ryan Michalik