Build value and insight from model validation activities

Financial services organizations can experience multiple challenges in their model validation and risk monitoring goals. Following are some common model validation challenges:

Resources and team experience

Success can be difficult for departments if their staffing levels aren’t sufficient to handle consistent model validation and risk monitoring.

Because of the COVID-19 pandemic, some organizations have had to manage with reduced department sizes or increased workloads without new hiring. Additionally, staff familiar with the model validation process might have moved on, resulting in critical knowledge gaps.

Risk-based testing

Applications of tools and models can vary across organizations. The risk-based testing deployed on each tool or model also varies depending on each organization’s decisions.

Organizations should establish confidence in the definitions and validation procedures associated with their model validation programs. They should not hesitate to discuss their approach with regulators or consultants with extensive experience in model validation.

A policy of waiting for advice or recommendations from regulators can contribute to issues in models being discovered later than they could have been.

Vendor collaboration

Regulators continue to have high expectations for use and independent testing of vendor models. As a result, financial services organizations maintain the responsibility of understanding their vendor models’ functions and the outputs received from them. As model technology evolves in areas such as artificial intelligence and machine learning, the need to stay updated grows more urgent.

When a model owner or vendor is not fully engaged, the resulting gaps in knowledge and protocols could lead to confusion and difficulty meeting regulatory demands down the road.

Documentation

Many model validations identify the need for more thorough documentation. Proper documentation of model inputs, user access, model output, and the approach to ongoing monitoring is critical to detecting foundational problems with models and making informed decisions about how to resolve them.

Ideally, an organization should not wait until a model is fully implemented to build a monitoring and validation plan. Policies for documentation should be established as soon as reasonably possible.

New acquisitions and partnerships

When financial services organizations take on new relationships, the responsibilities for models might fall under them. This situation can be especially common with fintech partnerships.

Fintechs and regulated financial services organizations don’t share the same requirements for model validation. For example, when a fintech arrives as part of an acquisition or partnership, the responsibility for its models often shifts to the financial services organization. Therefore, an organization must determine what risks it is – or could be – taking on with the new business relationship.