Model Risk Management

IA: How did model risk management become prominent in banking, and why is insurance only now catching up?

MRM expert: There is an important regulatory driver in the evolution of model risk management practices for banks. Basic model risk controls were formally introduced for market risk measurements by Basel I in 1996 and subsequently extended to credit risk models with Basel II in 2008. It was, however, the US Federal Reserve’s introduction of principles for model risk management in 2011 (SR11-7) that elevated model risk to a distinct category, envisaging a full spectrum of controls, from model discovery and inventory to risk assessment and reporting.

Several other jurisdictions introduced similar principles over the years, with PRA SS1/23 and the EU AI Act being the latest examples. The lag in implementation for insurance may simply be reflecting historical events that have led regulators to focus on this type of risk for banks sooner, e.g., 2008-10 financial crisis.

LR: One of the questions that clients ask about model risk management is about robust model definitions to capture more than regulatory models. Do you have any tips for insurers on how all-encompassing a definition should be?

MRM expert: There is not a single model definition that suits all organisations, but fortunately, regulators recognise this. The PRA’s definition in the supervisory statement is a necessary starting point, and each firm would need to define customised implementation guidelines to guide the model discovery and initial inherent risk classification process. For what falls outside the remit of a Model Risk policy, it is important to demonstrate that the risks attached to those analytics are managed under proportionate controls, e.g., an End User Computing policy, and other relevant measures. Being able to demonstrate that risks are adequately assessed and well managed, regardless of what labels we attach to them, is more important than adopting an all-encompassing definition.

IA: In your experience, how does model risk management add value? Do you have any practical tips to convey value to senior stakeholders?

MRM expert: The literature is now full of examples of market failures caused by errors in models (e.g., JPMorgan London Whale) or by inappropriate controls around the use of models or algorithms (e.g., Knight Capital demise). Only the most impactful model risk events appear in the news, but there would be many other instances of model failings causing lesser damage that are only reported internally or even go undetected, yet cumulate over time.

For some models, accuracy is a matter of both regulatory compliance and adherence to accounting principles of reporting and failings. For others, errors or misuse may impact both a firm’s reputation (e.g., bias in Amazon AI recruiting tools) and retail customer outcomes (e.g., unfair termination of bank accounts due to faulty fraud detection models). Poor model risk management may eventually lead to economic loss.

The more pervasive the models are in decision making, the more important it is to manage the risk of loss by introducing proportionate, albeit costly, controls. Regardless of whether there is an explicit regulatory expectation on model risk management, the decision to embark on developing a model risk management framework depends on the materiality of this risk, as all material risks should be managed and mitigated.

LR: What are your views on the challenges of implementing PRA’s SS1/23 in banking? Any lessons learned that you would share for those embarking on developing a model risk management framework for the first time?

MRM expert: PRA’s SS1/23 should not be challenging for firms that have adopted SR11/7 already. It represents an expansion of scope in that it explicitly calls for complex, material deterministic algorithms to be captured within model risk-type controls. The PRA’s supervisory statement also has explicit references to the interaction between model risk management and responsibilities with the UK senior management regime. In general, it adds granularity in terms of expectations on board awareness and reporting, inventory content, development and validation activities, as well as the management of model weaknesses.

There are a few key areas to get right from the start.

• Model discovery and inherent risk assessment processes, as this will allow firms to ‘size’ the problem and support.
• Minimum data, documentation, development, and validation requirements, including frequency of the periodic monitoring activities.
• Resources and hiring across the three lines of defence to support remediation and BAU cyclical activities.
• Importantly, an aggregate, although basic to start with, risk profiling and reporting process to monitor remediation, any unmitigated risk, and guide prioritisation of mitigation.

Taking a progressive agile approach to framework development and embedding is key to success, as the size and composition of the model inventory tends to change as the organisation refines the definition of a model and learns to implement it. Educating about model risk across all levels of seniority is crucial for successful implementation. The literacy requirements on AI for those firms caught in the EU AI Act compliance make the case for investing in training for regulatory compliance.

IA: How did the banking industry succeed in distinguishing itself from the more onerous governance and controls that apply to advanced approaches (internal models)? 

MRM expert: Essentially, by adopting a risk-based approach. This is about designing a robust approach to inherent risk assessment based on model materiality and complexity indicators, which can then inform the proportionality of the controls applied. Naturally, models impacting regulatory reporting and financial reporting would be classified as higher materiality. But similarly, those materially impacting retail customer outcomes, market conduct, and fraud or AML detection can be associated with higher materiality assessments. Complexity, as in the case of AI, can amplify the inherent risk assessment of a modelled solution.

Materiality essentially reflects the impact of the models on a firm’s business, products, and processes. The inventory and the tiering associated with it are a picture of what is important in an organisation's business model. Controls should be more or less demanding to reflect what matters to an organisation, what is key to its success and survival. Model identification and materiality assessment would also require bottom-up and top-down perspectives to ensure an appropriate balance of proportionate control environment for model risk.

LR: Can you give us an indication of the timelines for developing a workable system for model risk management? Can it be completed in one project phase, or is it an iterative process that can take time to refine?

MRM expert: The process can take more or less time depending on the size of the model inventory, the materiality of the models, and the tolerance for risk set by the board and embedded in a firm’s policy and standards.

The model discovery and initial risk assessment process can take months, as it typically requires discussion and engagement with both business operators and senior managers with a view to educating them as to what a model is and enabling them to disclose the list of models they own or operate. This involves creating an inventory of official models.

This phase also requires a desktop review of documents reliant on numbers and the re-tracing of the original source, being either a data point, a model (including expert judgements), or a deterministic quantitative process.

The definition of minimum policy requirements and remediation planning can run in parallel, but the need to hire resources to implement the plan means the formulation of a plan and related governance and senior buy-in can take up to a year for a global, complex organisation.

Remediation can take several years, depending on the size of the inventory and the resources deployed to document and validate the models. A highly material model can take, on average, one month for one FTE to be documented and three months for two FTEs to initially validate at a high standard.

IA: How does model risk management need to adapt to AI models?

MRM expert: This is a very current topic that is being debated in several industry forums right now, and there is no established best practice yet. AI solutions are powered by models that can and should be governed similarly, this is also what PRA SS1/23 explicitly says.

AI does not create new ‘types’ or broad categories of risks to be added to a typical risk taxonomy; however, the way the risks combine and manifest is new, so standards for managing AI-generated risks require adaptation. An AI risk oversight may call for centralised governance with representation from key risks. For instance, model risk, but also IT risk, compliance, and legal risks. The EU AI Act is also now enforceable for entities captured within its scope.

Central oversight is key, especially to ensure consistency in risk assessment and prioritisation, as well as the development of risk management practices in the early stages of deployment. In addition, the complexity and opaqueness of AI workings can make the ‘use risks’ more acute; the governance needs to have a very specific focus on the analysis of model behaviour and of the model’s limitations to support the user’s experience and develop trust. Establishing ethics principles to guide the deployment and usage of AI is also best practice, and there are several benchmarks on AI ethics now available in several jurisdictions.

The key driver of AI implementation is business acceleration and more intelligent use of human resources, and so an understanding of model accuracy in comparison to what a human can do becomes part of the initial formulation of the business case for deployment and may need to be independently challenged by model risk experts.

LR: To what extent is the model risk management approach developed by banks transferable to insurance?

MRM expert: My experience of designing and implementing an MRM framework outside of banking is that the principles of MRM are fully transferable, and it is simply a matter of adapting it to a different model taxonomy that reflects a specific business model and regulatory context. The SME for validating insurance-specific models may be scarce to begin with, and this is likely to be the most important challenge for the implementation of MRM controls in an insurance setting.

How can Crowe help?

As insurers face increasing regulatory scrutiny and operational complexity, the lessons from banking’s MRM frameworks offer a valuable blueprint. From defining models and assessing inherent risk to embedding proportionate controls and adapting to AI-driven challenges, the path forward requires a strategic, agile, and well-resourced approach. The importance of education, governance, and cross-functional collaboration cannot be overstated.

Crowe’s Consulting team brings deep expertise in both risk management and actuarial modelling, offering tailored support to help insurers build resilient, compliant, and future-ready MRM frameworks. Whether you're just beginning your MRM journey or refining an existing framework, Crowe can help you navigate the complexities with confidence and clarity.

For more information, contact Isaac Alfon and Lloyd Richards.