On June 21, 2019, Canada’s Office of the Superintendent of Financial Institutions (“OSFI”) released Draft Guideline E-25: Internal Model Oversight Framework (the “Draft Guideline”), which applies to the risk oversight frameworks that property and casualty insurers create and apply to the internal models that certain of them develop to comply with OSFI’s regulatory capital requirements under the Minimum Capital Test (MCT) Guideline.
It is important to note (i) that the focus of the Draft Guideline is on how insurers deal with the risks inherent in developing and deploying their internal models, rather than on the substance of the models themselves; and (ii) that the Draft Guideline affects only those insurers that use an internal model, which requires OSFI approval.
Comments will be received on the Draft Guideline until August 30, 2019. OSFI intends to issue the finalized Guideline by the end of 2019.
The Internal Model Risk Control (IMRC) Process
The core of the Draft Guideline is a detailed framework for the assessment of oversight procedures. The framework requires the establishment of Internal Model Risk Control (“IMRC”) processes to deal with internal model risk and data risk, including the designation of a Risk Control Officer or Committee (“RCO/C”).
Process for internal model risk
The IMRC process for internal model risk consists of:
- Audit trail of actions relating to the validation and use of the model;
- Security features to prevent unauthorized tampering with the model;
- Objective validation and vetting of the model and its results;
- Assessment of the controls over model risk; and
- Processes for tracking issues and concerns relating to the model or its use.
Process for data risk
A compliant IMRC process will focus on the “appropriateness, accuracy, completeness and timeliness” of data used in the internal model, and should include:
- Data quality assessment, including fitness for use, etc.;
- Data quality monitoring, including regular review of the performance of data collection, storage, transmission and processing systems;
- Identification and resolution of problems relating to the data in the system; and
- Identification of limitations relating to the data.
Risk control officer/committee
The RCO/C is responsible for vetting the effectiveness of model risk control processes, both when they are introduced and going forward. The RCO/C should be located in Canada and must have sufficient authority to challenge and, if necessary, effect change in the internal model. He or she should report directly to a member of the Board (or to another person with direct access to the Board), rather than to an individual on the business side such as a member of the model development team.
The Draft Guideline deals extensively with documentation requirements. Key aspects of these requirements include documenting the instructions that the insurer provides to internal users with respect to setting parameters and running internal models, as well as ensuring that vetting and validation processes are documented and that electronic records from their systems are retained (including any error messages that are generated during internal model runs).
All IMRC-related documentation should be reviewed and updated on a regular basis.
Application of the IMRC Process Through a Model’s Life Cycle
The IMRC process applies to all phases of an internal model’s life cycle. The Draft Guideline provides significant guidance with respect to OSFI’s expectations at every stage:
Phase 1: Initial development or subsequent modification
For new internal models, the Draft Guidance recommends that an economic or business rationale be identified and that the supporting evidence be documented. For both new and modified models, OSFI is proposing to require documentation of the reasons for the choice of a model, as well as of the process that will be followed as the model is developed. Such documentation should include descriptions of:
- The modelling technique that has been adopted;
- Any assumptions or approximations that have been relied on, with justifications;
- The data sources and data proxies that have been used; and
- Relevant weaknesses and limitations of the model, if any.
With respect specifically to model modifications, insurers should ensure that there are clear guidelines respecting the materiality of a modification, as well as a process for managing and documenting any modification that is implemented. In the case of modifications, it will be particularly important to focus on the controls that govern authorizations to make changes, as well as on empirical testing with respect to the effect of such modifications on results. The Draft Guideline cautions that OSFI approval may be required for some modifications.
Phase 2: Objective vetting
The RCO/C should vet the internal model or proposed modification prior to its implementation. This process should include:
- Reviewing the rationale and the evidence cited by the model development team in approving the model;
- Verifying that all documentation is current and available;
- Assessing the selection of the model or modification, relative to other options;
- Evaluation the three components – inputs, computation processes and reporting processes – of the development process; and
- Technical assessment of the model with respect to predictive capacity over a range of assumptions.
Once the vetting process is complete, the RCO/C should issue a recommendation to accept or reject the model. The RCO/C may also decide to approve a model while attaching specific conditions to its use.
Phase 3: Approval or rejection by the model executive
Once the RCO/C’s recommendation has been received, the insurer must issue a final approval or rejection, together with any limitations on the use of the new or modified model. Responsibility for this decision lies with the “Model Executive”, an internal group that must be separate from the group that is responsible for developing the model and also from the RCO/C.
Phase 4: Ongoing and objective validation
Ongoing and objective validation includes reaffirming the completeness of existing documentation as well as the assumptions and choice of data. The RCO/C is responsible for performing an objective validation of the model on at least an annual basis, in which it determines whether:
- The processes are being carried out correctly;
- The model is still fit for use;
- Results are being explained correctly;
- Results are consistent with expectations; and
- Tracked issues are being addressed.
The RCO/C is also required to conduct a benchmarking analysis, reexamine known limitations of the model, back-test the model results and perform a sensitivity analysis. The RCO/C should then document its findings and recommendations and inform senior management and the model executive. The insurer should have a process for tracking and assessing the response to the RCO/C’s findings and recommendations.
Phase 5: Decommissioning
Decommissioning is a potential final phase in the life cycle of an internal model, with respect to which insurers should have policies and procedures in place. Prior notification of OSFI and other stakeholders is important when an insurer chooses to decommission a model.
Internal Audit’s Role in Reviewing the IMRC Process
Finally, OSFI expects that insurers will establish requirements with respect to the periodic review of their internal model oversight by an internal audit team that is completely distinct and independent from the group that developed the internal model. The internal audit review and assessment should evaluate:
- Effectiveness of controls for model risk;
- Effectiveness of the IMRC process; and
- Accuracy and completeness of related reports and documentation.
As noted above, OSFI will be accepting comments on the Draft Guideline until August 30, 2019.
The content of this article is intended to provide a general guide to the subject matter. Specialist advice should be sought about your specific circumstances.