May 20, 2020
Anamaria Pieschacon, PhD
Director
Model Validation
Assessing Model Risk with Effective Validation
Olga Loiseau-Aslanidi , PhD
Director
Business Analytics
Petr Zemcik, PhD
Senior Director
Economics & Business Analytics
Effective Validation Webinar – May 2020 2
Moody's Analytics operates independently of the credit ratings activities of Moody's Investors
Service. We do not comment on credit ratings or potential rating changes, and no opinion or
analysis you hear during this presentation can be assumed to reflect those of the ratings agency.
Effective Validation Webinar – May 2020 3
Presenters
Olga Loiseau-Aslanidi PhD, Director
Head of APAC Risk Modelling
Economics & Business Analytics | Singapore
Petr Zemcik PhD, Senior Director
Head of EMEA Risk and Economics
Economics & Business Analytics | UK
Anamaria Pieschacon PhD, Director
Global Head of Model Validation
Economics & Business Analytics | USA
Effective Validation Webinar – May 2020 4
1. Model Risk in Spotlight
2. Effective Model Validation
3. Application to IFRS 9 Model
4. Key Takeaways
Agenda
1
Model Risk in Spotlight
Effective Validation Webinar – May 2020 6
Institutions Rely on Models to Guide Decisions
Manage risk, identify opportunities and comply with regulation
Collection &
Recovery
Business
&
Strategic Planning
Application Scorecards
Credit Policies
Risk Based Limit Management and Pricing
Risk and Profitability Based Decisioning
Credit Line Assignment
Risk Appetite Framework
Behavioral Scorecard
Credit Transition Matrix
Credit Line Management
Fraud Detection
Loss Forecasting
Scenario Generation
Stress Testing
Early Warning Indicators
Propensity and Churn Modeling
Scenario Generation
Stress Testing
Reverse Stress Testing
IFRS 9 Impairment Modeling
ICAAP with IRRBB
Credit Risk Concentration
Economic and Regulatory Capital
Collection Scorecard
Optimal Workout
Credit Policies
Roll Rate Analysis
Tracking Collectors Efficiency
Regulatory
Reporting
Origination
Portfolio
Management
Collection &
Recovery
Effective Validation Webinar – May 2020 7
Assess models’ stability and
validity
Timely and consistent model
adjustments such as recalibration
using most recent data, overlays
Incorporate regulators’ mitigating
actions
Enhance model monitoring
Identify most vulnerable
exposures
Planning for vulnerable
exposures and portfolios
under stress
Optimize capital
allocation
Beware of potential model
failures and model
interdependencies
Quantify what COVID-19
means for the economy
Generate multiple future
paths to revise existing
adverse scenarios
Which models I should be
most worried about?
Which aspects of models
are most affected?
Credit risk and liquidity
risk models are most
vulnerable
COVID-19 Calls for Model Revision
Mitigating model risk is a basis for effective crisis management
Affected Models in Scope
Changes in Market Conditions
Validation and Benchmarking
Portfolio Management
Understand Identify Enhance Act
Proactive Overhaul of Model Risk Management
Effective Validation Webinar – May 2020 8
Credit Risk Models Are Among Most Vulnerable
Need to improve model resilience during pandemic and beyond
Application,
behavioral,
transactional,
alternative data
Scorecards
Dynamic models
IFRS9, stress testing
PD | LGD | EAD
PD | LGD | EAD
IRB models
Pricing
Macroeconomic
Scenarios
COVID-19
Impact
2
Effective Model Validation
Effective Validation Webinar – May 2020 10
Robust Model Governance as a Precondition for
Effective Model Risk Management
Greater Model
Complexity
Amplified
Supervision
Significant
Financial Impact
Increased Data
Availability
More
Models
Effective Validation Webinar – May 2020 11
Effectiveness depends on a
combination of incentives,
competence, and influence
Managing Model Risk Involves Effective Challenge of Models
Effective Model Validation
Critical analysis
by objective
Identify
model
limitations
Effective Validation Webinar – May 2020 12
Intensity should be proportional to the materiality of the portfolio
Depth of Validation
Complexity
Materiality
Internal
Validation
External
Validation
In all cases,
» MRM team should establish model performance
thresholds for periodic monitoring.
» MRM team should run periodic performance tests
and perform formal annual validation.
MRM team can hire external validation
if they lack in-house expertise
Effective Validation Webinar – May 2020 13
3 Pillars for Effective Model Validation
Independence
Purposeful
Rigor
Expertise
Effective Validation Webinar – May 2020 14
Expertise & Purposeful Rigor
Other Advisory
Services
Gap Analysis, Best Practices
and Model Governance
Regulatory Capital &
Stress Testing
Models
Basel, CCAR, PRA, EBA etc.
Financial
Reporting
IFRS 9 and CECL
Business &
Strategic Planning
Models
Credit Policy, Marketing,
etc.
Loan Lifecycle
Management
Models
Application, Pricing,
Origination, Monitoring, Loss
Mitigation, Disposition
Credit Portfolio
Management
Models
Risk Appetite, Concentration
Risk, Counterparty,
Operational, etc.
Effective Validation Webinar – May 2020 15
» Model developers and owners
should coordinate all stages of
model lifecycle, including
implementation.
» Validators should provide
effective challenge to existing
models, based on purpose and
materiality.
» To avoid conflicts of interest,
validation should be performed
by a team independent from
model development.
Independence
Board
Board Risk Committee
Model Risk Committee
1
st
Line
Model Owner Modeler
Implementation
Manager
2
nd
Line
Validation and
Ongoing
Monitoring
3rd Line
Internal Audit
Effective Validation Webinar – May 2020 16
Our Validation Process
Qualitative
Replication and outcomes
analysis
Validation Report
Comparison of inputs and
outputs of estimates from
alternatives allows to assess
and manage model risk
Evaluation of conceptual soundness
Assessment based on the
qualitative, quantitative and
benchmarking analysis
Benchmarking
Quantitative
Effective Validation Webinar – May 2020 17
Model Evaluation – Action Ratings
Satisfactory
The model has no critical
findings and is suitable for
deployment.
Satisfactory with
Recommendations
The model’s performance is
satisfactory and is suitable
for deployment.
Nevertheless, the validators
have identified areas where
the model could undergo
improvements that may
improve its overall
performance.
Needs Improvement
The validators have
identified multiple critical
findings that have a negative
impact on the model’s
performance. The current
model provides at least a
minimally adequate level of
performance and can be
used in its present form.
Unsatisfactory
There are important flaws in the
model’s underlying data,
conceptual framework, or
development process. Either i)
the model cannot perform its
intended function and should
not be used in any decision-
making capacity, or ii) there is
not enough evidence to show
that the model can perform its
intended function and it should
not be used in any decision-
making capacity until such
evidence becomes available.
Effective Validation Webinar – May 2020 18
Issues Identified and Recommended Actions – Generic Example
Final Assessment: Model Ratings by Category
Risk Category Rating Comments
Documentation The documentation needs to include XYZ.
Data Cleaning and Treatments …
Variable Selection Process …
Model Selection …
Model Performance …
Sensitivity Analysis
Model Replication …
Monitoring and Performance Tracking …
Overall Rating
The report will explicitly describe that the above risk categories do not hold equal weighting. The categories
shown may not reflect actual categories used.
Effective Validation Webinar – May 2020 19
Our Validation Process
Model document
review/understanding
Evaluate
» Purpose, scope, materiality
» Model selection process
» Data, conceptual soundness
» Assumptions & limitations
» Uncertainty & mitigating controls
» Review model governance,
ongoing monitoring/tests
Replication
Review and verify additional
analysis submitted by model
owners
In-sample and out-of-sample
performance evaluation
Push documents and scripts to
production
Discussion with model
owners/stakeholders
Stability and robustness
Sensitivity Analysis
Identify and discuss any gaps with
stakeholders
Initial model assessment
» Qualitative commentary on
possible model deficiencies,
implementation errors
» Categorize by severity and
issue recommendations
» Independent analysis
» Independent implementation
» Commentary on identified
shortcomings
» Final document with action
ratings
» Recommendations and
summary
of findings
COMPONENTS
DELIVERABLES
Qualitative
Validation
Quantitative
Validation
Consolidation
Preliminary
Model Review
01 02 03 04
Benchmarking*
Document and categorize the
findings by severity, issue
recommendations
Effective Validation Webinar – May 2020 20
We Measure Model Risk by Benchmarking
3
Application to IFRS 9
Models
Effective Validation Webinar – May 2020 22
Impairment Model
Macroeconomic Scenario Forecast
Scenario Probability Weights
Probability of
Default
Survival Probability
Loss Given
Default
Exposure at
Default
Discount Factor
Expected
Credit Loss
X X X =
Behavioral
Component
Probability of Cure
Effective/contractual
Interest Rate
Unbiased Point-in-Time Estimates
Stage
1, 2 or 3
IFRS 9: Macroeconomic Scenarios & Expected Credit Loss Calculation 23
An Integrated Process
Credit Risk Models
Credit Risk
Models
Macro-
economic
Scenarios
Data
Expected
Credit Loss
Model
Sensitivity
Analysis
Implementation
Results and
Reports
Portfolio Data
Macroeconomic
Scenarios
IRB Models / Basel
Models
Stress Testing
Models
IFRS 9 Models
April 2020 24
Scenario Severity Shift
Source: Moody’s Analytics
BL Apr
S3 Apr
BL FebS3 Feb
S1 Feb
S1 Apr
Effective Validation Webinar – May 2020 25
PD Modelling Approaches
Segment level
Account level
PD= f
Lifecycle
Quality of Vintage
Forward-looking Indicator
Dynamic evolution of vintages as they mature
Variable capturing the heterogeneity across cohorts:
vintage dummies, portfolio characteristics and/or
economic conditions at origination
Sensitivity of performance to the evolution of
macroeconomic and credit series
Modelling approach with three key factors influencing vintage
segment performance:
PD is forecasted using customer and loan characteristics,
and macroeconomic indicators using panel data
econometric techniques
PD= f
Customer and Loan Level Characteristics
Macroeconomic Drivers
Characteristics such as LTV, score, months on book,
education, etc.
Select pre-macro model using single factor and
multifactor analysis
Variable selection algorithm to select macroeconomic
drivers.
1. Segmentation
» Switch to bucketing based on DPD & LTV
» No further segmentation
» Internal portfolio
» Macro data
» Initial estimation
» Smoothing
» Scaling
Transition Matrix Approach
2. Data Inputs
3. TTC Matrix Creation
Effective Validation Webinar – May 2020 26
Looking at Forecast Properties
» Policy variables, e.g. CPI
» Changes in past correlations
» Non-cyclical sectors
» Growth rates:
– Low range level variables,
e.g. RMM
– QoQ growth rate
PD & Driver
Correlation
Driver
Forecasts
PD
Forecasts
Issue
Inconsistent
Volatile
No
convergence
» Long-term forecast property
of transformation
Downside
Upside
Upside
Downside
Upside
Upside
Upside
Upside
Downside
Downside
Downside
Downside
Driver
PD
Time
Time
Time
Time
Time
Time
Driver
PD
Driver
PD
Macro Driver
Macro DriverMacro Driver
PD
PD
PD
Effective Validation Webinar – May 2020 27
LGD Design Approaches
Balance and Recoveries
For a facility i, time t and workout period w:
= 1 −
,
−
,+
,
By Assumption
LGD of 50-60% for PF, 30-40% for RE and
65-75% for CC; fully insured products usually
get LGD of 5-10%.
Estimates of recovery costs range from 1-2%.
Default Vintages & Macro Drivers
Roll Rate Modelling
= 1 −
Effective Validation Webinar – May 2020 28
EAD Design Approaches
Fixed Term Products - Amortization Revolving Products - CCF
,+ℎ
=
,+ℎ
+ ∗
,
Credit Limit
Time
Balance
EAD
Undrawn amount x CCF
Drawn Amount
Effective Validation Webinar – May 2020 29
Evaluation of SICR
Quantitative Approach
Characteristics of the metric:
» Forward-looking (scenarios)
» Capture risk of default
» Lifetime information
» Available at origination and at reporting date
Status Criteria Stage
Non-Default Lifetime PD(T) ≤ Lifetime PD
0
(T) + Buffer 1
Non-Default Lifetime PD(T) > Lifetime PD
0
(T) + Buffer 2
Default 3
What is the optimal d to identify SICR?
» Buffer is the optimal value of d that maximizes an
accuracy ratio from good:bad odds analysis
» We examine differences (in logit) between
– the lifetime PD at the reporting date Lifetime PD(T)
– the lifetime PD at the same age as the reporting date
forecasted at origination Lifetime PD
0
(T)
for different historical reporting dates
Qualitative Approach
» DPD
» Forbearance
» Watch list
» …
.5
.55
.6
.65
.7
.75
Accuracy Ratio
0 .61 1 2 3
Buffer Size (logit)
Effective Validation Webinar – May 2020 30
= �
, ∗ , ∗ , ∗ ,
=
1
,
1
+
2
2
+ … +
(|
)
Probability-Weighted ECL by instrument:
ECL by scenario (s) & instrument (i):
ECL Calculation
Effective Validation Webinar – May 2020 31
IFRS 9 Validation Process
0
2
4
6
8
200 0m1 2005m1 2010m1 2015m1 2020m1 202 5m1
MA Baseline
0
2
4
6
8
200 0m1 2005m1 2010m1 2015m1 2020m1 202 5m1
MA St re ss
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0.09
0.1
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
PD
Time
Robustness & Sensitivity Analysis Report
Portfolio Behavior to Changing
Macroeconomic Conditions
Qualitative Quantitative
Final
Assessment
Methodology
Data use,
description &
treatment
Regulatory
compliance
Model
governance
Data analysis
Model
replication
Model
performance
Benchmark
model
development
Written report
Observation,
findings and
recommendati
ons and or
remedial
actions
Effective Validation Webinar – May 2020 32
IFRS 9 Case Study – Impact of COVID-19
Baseline Feb 2020
Baseline Apr 2020
IFRS 9 Stage # % Exposure ECL=0.03 IFRS 9 Stage # % Exposure ECL=0.05
1
92,090
99.00
8,275,327,246
0.00
1
92,047 98.90 8,266,730,875 0.01
2
717
0.83
69,352,356
0.89
2
760 0.93 77,948,726 1.68
3
146
0.17
13,986,747
12.21
3
146 0.17 13,986,747 12.82
Upside Feb 2020
Upside Apr 2020
IFRS 9 Stage # % Exposure ECL=0.03 IFRS 9 Stage # % Exposure ECL=0.04
1
92,093
99.01
8,275,597,498
0.00
1
92,082 98.99 8,273,841,624 0.01
2
714
0.83
69,082,104
0.79
2
725 0.85 70,837,977 1.31
3
146
0.17
13,986,747
12.21
3
146 0.17 13,986,747 12.82
Downside Feb 2020
Downside Apr 2020
IFRS 9 Stage # % Exposure ECL=0.04 IFRS 9 Stage # % Exposure ECL=0.07
1
92,079
98.97
8,272,959,874
0.01
1
91,770 98.34 8,219,603,740 0.02
2
728
0.86
71,719,727
1.35
2
1,037 1.50 125,075,861 1.88
3
146
0.17
13,986,747
12.21
3
146 0.17 13,986,747 12.81
Prob-weighted Feb 2020
Prob
-weighted Apr 2020
IFRS 9 Stage # % Exposure ECL=0.03 IFRS 9 Stage # % Exposure ECL=0.05
1
92,086
99.00
8,274,895,046
0.01
1
92,020 98.84 8,261,408,180 0.01
2
721
0.83
69,784,556
1.00
2
787 1.00 83,271,421 1.76
3
146
0.17
13,986,747
12.21
3
146 0.17 13,986,747 12.81
4
Key Takeaways
Effective Validation Webinar – May 2020 34
Proactive Overhaul of Model Risk Management
Understand Identify Enhance Act
Affected Models in Scope
Changes in Market Conditions
Validation and Benchmarking
Portfolio Management
moodysanalytics.commoodysanalytics.com
West Chester, EBA-HQ
+1.610.235.5299
121 North Walnut Street, Suite 500
West Chester PA 19380
USA
New York, Corporate-HQ
+1.212.553.1653
7 World Trade Center, 14th Floor
250 Greenwich Street
New York, NY 10007
USA
London
+44.20.7772.5454
One Canada Square
Canary Wharf
London E14 5FA
United Kingdom
Toronto
+1.416.681.2133
200 Wellington Street West, 15th Floor
Toronto ON M5V 3C7
Canada
Prague
+420.23.474.7500
Pernerova 691/42
186 00 Prague 8 - Karlin,
Czech Republic
Sydney
+61.2.9270.8111
Level 10
1 O'Connell Street
Sydney, NSW, 2000
Australia
Singapore
+65.6511.4400
6 Shenton Way
#14-08 OUE Downtown 2
Singapore 068809
Shanghai
+86.21.6101.0172
Unit 2306, Citigroup Tower
33 Huayuanshiqiao Road
Pudong New Area, 200120
China
Contact Us: Content Solutions - Economics & Business Analytics Offices
Effective Validation Webinar – May 2020 37
© 2020 Moody’s Corporation, Moody’s Investors Service, Inc., Moody’s Analytics, Inc. and/or their licensors and affiliates (collectively, “MOODY’S”). All
rights reserved.
CREDIT RATINGS ISSUED BY MOODY'S INVESTORS SERVICE, INC. AND/OR ITS CREDIT RATINGS AFFILIATES ARE MOODY’S CURRENT
OPINIONS OF THE RELATIVE FUTURE CREDIT RISK OF ENTITIES, CREDIT COMMITMENTS, OR DEBT OR DEBT-LIKE SECURITIES, AND
MATERIALS, PRODUCTS, SERVICES AND INFORMATION PUBLISHED BY MOODY’S (COLLECTIVELY, “PUBLICATIONS”) MAY INCLUDE SUCH
CURRENT OPINIONS. MOODY’S INVESTORS SERVICE DEFINES CREDIT RISK AS THE RISK THAT AN ENTITY MAY NOT MEET ITS
CONTRACTUAL FINANCIAL OBLIGATIONS AS THEY COME DUE AND ANY ESTIMATED FINANCIAL LOSS IN THE EVENT OF DEFAULT OR
IMPAIRMENT. SEE MOODY’S RATING SYMBOLS AND DEFINITIONS PUBLICATION FOR INFORMATION ON THE TYPES OF CONTRACTUAL
FINANCIAL OBLIGATIONS ADDRESSED BY MOODY’S INVESTORS SERVICE CREDIT RATINGS. CREDIT RATINGS DO NOT ADDRESS ANY
OTHER RISK, INCLUDING BUT NOT LIMITED TO: LIQUIDITY RISK, MARKET VALUE RISK, OR PRICE VOLATILITY. CREDIT RATINGS, NON-
CREDIT ASSESSMENTS (“ASSESSMENTS”), AND OTHER OPINIONS INCLUDED IN MOODY’S PUBLICATIONS ARE NOT STATEMENTS OF
CURRENT OR HISTORICAL FACT. MOODY’S PUBLICATIONS MAY ALSO INCLUDE QUANTITATIVE MODEL-BASED ESTIMATES OF CREDIT RISK
AND RELATED OPINIONS OR COMMENTARY PUBLISHED BY MOODY’S ANALYTICS, INC. AND/OR ITS AFFILIATES. MOODY’S CREDIT RATINGS,
ASSESSMENTS, OTHER OPINIONS AND PUBLICATIONS DO NOT CONSTITUTE OR PROVIDE INVESTMENT OR FINANCIAL ADVICE, AND
MOODY’S CREDIT RATINGS, ASSESSMENTS, OTHER OPINIONS AND PUBLICATIONS ARE NOT AND DO NOT PROVIDE RECOMMENDATIONS
TO PURCHASE, SELL, OR HOLD PARTICULAR SECURITIES. MOODY’S CREDIT RATINGS, ASSESSMENTS, OTHER OPINIONS AND
PUBLICATIONS DO NOT COMMENT ON THE SUITABILITY OF AN INVESTMENT FOR ANY PARTICULAR INVESTOR. MOODY’S ISSUES ITS
CREDIT RATINGS, ASSESSMENTS AND OTHER OPINIONS AND PUBLISHES ITS PUBLICATIONS WITH THE EXPECTATION AND
UNDERSTANDING THAT EACH INVESTOR WILL, WITH DUE CARE, MAKE ITS OWN STUDY AND EVALUATION OF EACH SECURITY THAT IS
UNDER CONSIDERATION FOR PURCHASE, HOLDING, OR SALE.
MOODY’S CREDIT RATINGS, ASSESSMENTS, OTHER OPINIONS, AND PUBLICATIONS ARE NOT INTENDED FOR USE BY RETAIL INVESTORS
AND IT WOULD BE RECKLESS AND INAPPROPRIATE FOR RETAIL INVESTORS TO USE MOODY’S CREDIT RATINGS, ASSESSMENTS, OTHER
OPINIONS OR PUBLICATIONS WHEN MAKING AN INVESTMENT DECISION. IF IN DOUBT YOU SHOULD CONTACT YOUR FINANCIAL OR OTHER
PROFESSIONAL ADVISER.
ALL INFORMATION CONTAINED HEREIN IS PROTECTED BY LAW, INCLUDING BUT NOT LIMITED TO, COPYRIGHT LAW, AND NONE OF SUCH
INFORMATION MAY BE COPIED OR OTHERWISE REPRODUCED, REPACKAGED, FURTHER TRANSMITTED, TRANSFERRED, DISSEMINATED,
REDISTRIBUTED OR RESOLD, OR STORED FOR SUBSEQUENT USE FOR ANY SUCH PURPOSE, IN WHOLE OR IN PART, IN ANY FORM OR
MANNER OR BY ANY MEANS WHATSOEVER, BY ANY PERSON WITHOUT MOODY’S PRIOR WRITTEN CONSENT.
MOODY’S CREDIT RATINGS, ASSESSMENTS, OTHER OPINIONS AND PUBLICATIONS ARE NOT INTENDED FOR USE BY ANY PERSON AS A
BENCHMARK AS THAT TERM IS DEFINED FOR REGULATORY PURPOSES AND MUST NOT BE USED IN ANY WAY THAT COULD RESULT IN THEM
BEING CONSIDERED A BENCHMARK.
All information contained herein is obtained by MOODY’S from sources believed by it to be accurate and reliable. Because of the possibility of human or
mechanical error as well as other factors, however, all information contained herein is provided “AS IS” without warranty of any kind. MOODY'S adopts all
necessary measures so that the information it uses in assigning a credit rating is of sufficient quality and from sources MOODY'S considers to be reliable
including, when appropriate, independent third-party sources. However, MOODY’S is not an auditor and cannot in every instance independently verify or
validate information received in the rating process or in preparing its Publications.
To the extent permitted by law, MOODY’S and its directors, officers, employees, agents, representatives, licensors and suppliers disclaim liability to any
person or entity for any indirect, special, consequential, or incidental losses or damages whatsoever arising from or in connection with the information
contained herein or the use of or inability to use any such information, even if MOODY’S or any of its directors, officers, employees, agents, representatives,
licensors or suppliers is advised in advance of the possibility of such losses or damages, including but not limited to: (a) any loss of present or prospective
profits or (b) any loss or damage arising where the relevant financial instrument is not the subject of a particular credit rating assigned by MOODY’S.
To the extent permitted by law, MOODY’S and its directors, officers, employees, agents, representatives, licensors and suppliers disclaim liability for any
direct or compensatory losses or damages caused to any person or entity, including but not limited to by any negligence (but excluding fraud, willful
misconduct or any other type of liability that, for the avoidance of doubt, by law cannot be excluded) on the part of, or any contingency within or beyond the
control of, MOODY’S or any of its directors, officers, employees, agents, representatives, licensors or suppliers, arising from or in connection with the
information contained herein or the use of or inability to use any such information.
NO WARRANTY, EXPRESS OR IMPLIED, AS TO THE ACCURACY, TIMELINESS, COMPLETENESS, MERCHANTABILITY OR FITNESS FOR ANY
PARTICULAR PURPOSE OF ANY CREDIT RATING, ASSESSMENT, OTHER OPINION OR INFORMATION IS GIVEN OR MADE BY MOODY’S IN ANY
FORM OR MANNER WHATSOEVER.
Moody’s Investors Service, Inc., a wholly-owned credit rating agency subsidiary of Moody’s Corporation (“MCO”), hereby discloses that most issuers of debt
securities (including corporate and municipal bonds, debentures, notes and commercial paper) and preferred stock rated by Moody’s Investors Service, Inc.
have, prior to assignment of any credit rating, agreed to pay to Moody’s Investors Service, Inc. for credit ratings opinions and services rendered by it fees
ranging from $1,000 to approximately $2,700,000. MCO and Moody’s investors Service also maintain policies and procedures to address the independence
of Moody’s Investors Service credit ratings and credit rating processes. Information regarding certain affiliations that may exist between directors of MCO
and rated entities, and between entities who hold credit ratings from Moody’s Investors Service and have also publicly reported to the SEC an ownership
interest in MCO of more than 5%, is posted annually at
www.moodys.com under the heading “Investor Relations — Corporate Governance — Director and
Shareholder Affiliation Policy.”
Additional terms for Australia only: Any publication into Australia of this document is pursuant to the Australian Financial Services License of MOODY’S
affiliate, Moody’s Investors Service Pty Limited ABN 61 003 399 657AFSL 336969 and/or Moody’s Analytics Australia Pty Ltd ABN 94 105 136 972 AFSL
383569 (as applicable). This document is intended to be provided only to “wholesale clients” within the meaning of section 761G of the Corporations Act
2001. By continuing to access this document from within Australia, you represent to MOODY’S that you are, or are accessing the document as a
representative of, a “wholesale client” and that neither you nor the entity you represent will directly or indirectly disseminate this document or its contents to
“retail clients” within the meaning of section 761G of the Corporations Act 2001. MOODY’S credit rating is an opinion as to the creditworthiness of a debt
obligation of the issuer, not on the equity securities of the issuer or any form of security that is available to retail investors.
Additional terms for Japan only: Moody's Japan K.K. (“MJKK”) is a wholly-owned credit rating agency subsidiary of Moody's Group Japan G.K., which is
wholly-owned by Moody’s Overseas Holdings Inc., a wholly-owned subsidiary of MCO. Moody’s SF Japan K.K. (“MSFJ”) is a wholly-owned credit rating
agency subsidiary of MJKK. MSFJ is not a Nationally Recognized Statistical Rating Organization (“NRSRO”). Therefore, credit ratings assigned by MSFJ
are Non-NRSRO Credit Ratings. Non-NRSRO Credit Ratings are assigned by an entity that is not a NRSRO and, consequently, the rated obligation will not
qualify for certain types of treatment under U.S. laws. MJKK and MSFJ are credit rating agencies registered with the Japan Financial Services Agency and
their registration numbers are FSA Commissioner (Ratings) No. 2 and 3 respectively.
MJKK or MSFJ (as applicable) hereby disclose that most issuers of debt securities (including corporate and municipal bonds, debentures, notes and
commercial paper) and preferred stock rated by MJKK or MSFJ (as applicable) have, prior to assignment of any credit rating, agreed to pay to MJKK or
MSFJ (as applicable) for credit ratings opinions and services rendered by it fees ranging from JPY125,000 to approximately JPY250,000,000.
MJKK and MSFJ also maintain policies and procedures to address Japanese regulatory requirements.
