Why model risk management is good for business

Insurers rely heavily on models to support many business decisions ranging from pricing to capital management, and including reserving, reinsurance, and asset valuation. How these models are managed to ensure that business decisions are supported appropriately is an important business challenge, which is only increasing as models grow in terms of scope and complexity.

Some insurance models, e.g., capital models, are subject to intense external scrutiny through regulatory model approval and oversight. The Prudential Regulation Authority (PRA) has signalled its intention to turn its attention to other models used in insurance.

For those insurers that have not formally considered model risk, this is likely to become a game changer impacting risk functions and model owners. The crucial ingredient to meet this challenge is encapsulated in the often-quoted statement from the statistician George Box that “all models are wrong, but some are useful” and that it is important to be  “… alert to what is importantly wrong” by understanding where the model is right-enough to be useful: “It is inappropriate to be concerned about mice when there are tigers aboard.”

For insurers, this means there is a need to understand specific model risks within their business and prioritise appropriate remedial actions.

Model risk management as a regulatory concern

The PRA recently published its requirements for model risk management for banks (SS1/23), setting out its expectations for model development, validation, and governance, as well as management’s understanding of complex models, see Figure 1 for a summary of the principles.

Figure 1: The PRA’s model risk management principles

The PRA has also highlighted in its latest business plan that this would be a key area of focus for banking supervision. While currently only applicable to banks, the PRA has asked insurers to consider which elements of the policy are relevant to them in the 2023 priorities letter.

In its business plan the PRA made clear that it will continue to assess the application of these principles to insurers. More recently the PRA clarified that given the ongoing Solvency II review, it decided not to extend the proposals to insurers at this point in time. The PRA “intends to consider at a later stage whether there is a need to strengthen model risk management practices for insurers.” Our view is it is more than likely to be a question of when, not if, insurers must consider model risk from a regulatory perspective beyond the requirements of Solvency II.

The case for insurers’ early engagement with model risk

We think there are two main reasons for insurers to think early about model risk.

Model risk is material for insurers, and is growing

Solvency II prompted a significant investment in controls and governance for capital models and bolstered the use of such models through wider business understanding. As a result, capital models and the data that feed them typically go through a high degree of technical accuracy testing. However, such scrutiny is often not replicated in models that are crucial to the running of the business, including pricing, reserving, and internal rating models, even where such models feed the well documented capital model. In this context, model risk management is therefore both a technical and business challenge.

At the same time, the use of models and their complexity is increasing.

• The growing use of machine learning in insurance creates its own set of issues with model reproducibility, understanding and challenge, and data ethics; particularly relevant in light of consumer duty rules, where such models affect consumer outcomes.
• Climate metrics are moving from being a directional tool to be increasingly used in decision-making. The more this happens, the more important it is that boards understand the limitations of the climate models that create these metrics.

The main model risk exposure tends to arise from the model scope and how risks evolve.   Current heightened inflation is a case in point where model risk arises from the limited inclusion of inflation in models built during the decades of low inflation. In these types of scenarios, one would expect effective model risk management to lead to a timely response and adjustment of models to reflect a changed inflationary environment.

There are several documented examples of insurers getting into difficulties because of crystallisations in model risk:

• the Australian Securities and Exchange Commission has recently commenced litigation against two subsidiaries of IAG, alleging that IAG’s pricing algorithm has led to misleading customers. The court explicitly called out that robust systems and controls are needed, especially where complex pricing systems and algorithms are used
• JP Morgan’s billion-dollar losses in 2012 were partly the result of an error prone risk modelling system, including volatility calculations that required data to be cut and pasted between spreadsheets, later compounded by traders concealing losses using the same flaws
• a meta-analysis of insurer failures conducted by Canada’s Property and Casualty Insurance Compensation Corporation, found that insurers entering new lines of business regularly struggle to adequately price for risks owing to incorrect model assumptions, and that this is one of the leading causes of new insurer failures.

Developing proportionate, business-relevant solutions for model risk takes time

The biggest danger for firms seeking to enhance their approach model risk is to fail to develop a proportionate approach to model risk, often by seeking to replicate the existing Solvency II Internal Model control environment across the business. An approach that fits the range of models a company uses requires tailoring, and a ranking of the models in question is key to ensuring that the controls are pitched at an appropriate level that encourages model development and innovation rather than stifling it.

Our experience is that when companies are under pressure to meet regulatory deadlines, it can lead to an ineffective ‘tick box’ type approach. While these approaches can ensure that regulatory expectations are met, they do not typically contribute to the effectiveness and efficiency of risk management, which in this case requires a proportionate and business-relevant solution.

Where to start with model risk management?

For firms to understand and manage their full universe of risks effectively, they need to be able to manage the risks created by using models to support decision-making. Doing so is, simply, good risk management practice.

The PRA’s model risk management Principles (see Figure 1) represents a useful structure within which to think about model risk. We would encourage insurers to consider as a first step the roles and responsibilities of business owners and risk function in respect of the firm’s overarching approach to model risk, to agree where responsibility sits for defining and challenging the approach vs implementing mitigation.

Developing an inventory of models that includes an appropriate risk ranking is a crucial next step to understand an organisation’s model environment and begin to determine where the firm should focus its attention on conducting full risk assessments of individual models.

A risk assessment of models might seem like an insurmountable challenge unless it is approached in a pragmatic and risk-ranked manner.
Our experience of supporting insurers with their approach to model risk has identified internal and external drivers that represent both opportunities and challenges towards the management of models, set out in Figure 2.  

Figure 2: internal and external opportunities and challenges of models

Models are an integral part of how insurance companies are run. Although their development and use require specialist knowledge, that does not mean they should be excluded from good management disciplines.  Managing model risk makes good business sense.

Insurers have to date not felt the same regulatory pressure that banks have, however the PRA is clear in its business plan that it will continue to assess the application of model risk principles to insurers. Establishing practices and processes early on can reduce the cost of applying them whilst ensuring better business outcomes, and ultimate help firms to get ahead of regulations.

Finally, and most importantly, better models yield better results! While it may be true that all models are wrong, it is inescapable that we use them to support or even take over business decision-making every single day, and ensuring decision makers know about the tigers, even if they miss a few mice, is essential to success.

How can Crowe help?

Crowe’s risk consulting team is experienced in risk management, actuarial modelling, and model approval. Our team has supported insurers with ensuring models are appropriately developed for new business lines, improving the efficiency and effectiveness of models, and supporting firms through the internal model approval process.

For more information contact Isaac Alfon and Lloyd Richards.