Morningstar Medalist Ratings: Improving Ratings Efficacy
We will make two methodology enhancements for the Morningstar Medalist Rating™, our single, encompassing forward-looking rating for managed investments.
Morningstar is pleased to inform you that we will be further enhancing our Morningstar Medalist Ratings from the 29 October 2024.
We will make two methodology enhancements for the Morningstar Medalist RatingTM, our single, encompassing forward-looking rating for managed investments:
- We will enhance the process by which we estimate how much excess return, or “alpha”, a managed investment of a given type can deliver before fees for both Analyst and algorithmically assessed funds.
- We will enhance the algorithmic approach that we employ when evaluating the Process pillar of passively managed equity vehicles that analysts do not cover.
Enhancement of Process for Estimating Pre-Fee Alpha
The Medalist Rating system currently relies on our Alpha Potential Estimate (APE) for a given Morningstar category to estimate how much value, or “alpha”, a manager is expected to add versus its assigned benchmark before fees, as outlined below. We multiply that estimate by the numeric values corresponding to the pillar ratings we assign to People, Process, and Parent, and those pillars’ weightings in the framework. We sum those products to arrive at an overall estimated pre-fee alpha for that managed investment.
From that estimate, we subtract the investment’s fees to derive an estimated net alpha, and it is that net alpha that determines the managed investment’s overall rating—which takes the form of Gold, Silver, Bronze, Neutral, or Negative--under the Medalist Rating framework.
While we are not revising the process by which we assign Medalist Ratings, we are changing the way we derive APE as follows:
Current APE Approach
Currently, we derive APE by calculating rolling 36-month regressions of each fund’s pre-fee returns versus its assigned benchmark, repeating for all funds in the peer group through its history (we include data as far back as 2000 where available). From this history, we form a distribution ranging from the lowest to the highest pre-fee alphas. To derive APE, we calculate the difference between the pre-fee alphas at the 25th and 75th percentiles of the distribution and then divide that difference in half. We have referred to this as the “semi-interquartile range”, which has served as the multiplier in the process described above.
Revised APE Approach
From the October 29, we will change the way we arrive at estimated pre-fee alpha, replacing the “semi-interquartile range” with a different approach that conveys more precise information about the potential likelihood and magnitude of a fund-adding value before fees, adjusted for risk. Specifically, the enhanced approach takes the shape and skewness of the alpha distribution into a fuller account.
Once we have formed a distribution of historical pre-fee alphas in the manner described under “Present APE Derivation” immediately above, we will divide the number of positive alphas in the distribution by the total number of observations. This ratio indicates the frequency with which funds have generated positive pre-fee alpha versus their benchmark index over time. We then multiply this ratio by the median positive alpha in the distribution, the median alpha representing the likely magnitude of positive excess returns based on history.
In many peer groups, the distribution of pre-fee alphas skews negative, with less than 50% of observations being positive. But because the semi-interquartile range focuses only on the historical dispersion of pre-fee alphas (represented by half the difference between the 25th and 75th percentiles of the distribution), it does not take this skewness into consideration. The enhanced approach will do so.
Impact to Ratings
We expect to lower the Alpha Potential Estimate in many peer groups where the distribution of pre-fee alphas skews negative. This will likely impact Medalist Ratings once we incorporate the reduced Alpha Potential Estimates into the calculation of net alpha, which determines a managed investment’s rating.
Given this, we expect a material number of ratings – approximately 13%, to change due to these updates, which are likely to be largely downgrades. Specifically, we expect the number of investments rated Gold, Silver, and Bronze to decrease.
Ratings Generated by Analysts
We expect the Medalist Rating change to be incorporated gradually into the Ratings process when covered by analysts. These ratings are updated by an analyst on their periodic review which will occur over the span of approximately 14 months post 29 October 2024.
Ratings not Generated by Analysts (Machine Learning Algorithm)
We expect the Medalist Rating change to be incorporated into the Ratings of all funds not rated by an analyst from 29 October 2024. These ratings are refreshed monthly as part of our machine learning process.
We will provide a more detailed impact analysis in future communications.
Changes to Algorithmic Process Pillar Assignment for Passively Managed Equity Vehicles
For passively managed vehicles, which we define as managed investments that track an index, the Process pillar accounts for 80% of the overall Medalist Rating, with People and Parent representing 10%, respectively.
We assign Process pillar scores in three ways:
- Directly, by the analyst (i.e., an analyst covers that vehicle),
- Indirectly, by the analyst (i.e., an analyst does not cover the vehicle concerned but covers another vehicle that tracks the same index), and
- Directly, by algorithm (i.e., an analyst does not cover the vehicle concerned or any other vehicles that track the same index).
Under this third scenario, we are changing the way we assign Process Pillar ratings to passive equity vehicles. Please note that this change applies only to passively managed vehicles not classified as “strategic beta” (“Strategic beta” refers to managed investments that track an index that seeks to enhance returns or minimize risk relative to more traditional benchmarks). The change we are making is as follows:
Current Process Pillar Approach for Passive Vehicles
We assign Process pillar ratings based on an algorithmic assessment conducted by a random forest model. The random forest model is a generalized machine learning routine trained on the body of Morningstar analyst decisions. Specifically, the model evaluates the relationship between the ratings that Morningstar analysts have assigned to managed investments over time and the attributes of the managed investments assigned those ratings. The model then aims to apply those learnings to the universe of managed investments that analysts do not fully cover, associating managed investments with ratings based on their attributes.
In this way, the model seeks to emulate the way an analyst would conduct the evaluation if she were assigned coverage of that managed investment. It is worth noting, however, that this machine-learning technique is not tailored to particular types of managed investments, as it learns from the full corpus of ratings, many of which have been assigned to actively managed, not passively managed, vehicles.
Enhanced Process Pillar Approach for Passive Vehicles
From 29 October 2024, we will replace the random forest model used to assess the Process pillar with a rules-based system that we will use to algorithmically evaluate the Process pillar for passively managed equity vehicles. The new approach is tailored to focus on the most relevant and salient attributes to an analyst’s assessment of the index they are evaluating when covering a passive equity vehicle. In this way, the algorithmic approach will better align with the analysts’ approach take when assigning the Process pillar rating to passive stock vehicles they cover.
We have conducted back-testing to validate this enhanced approach, finding that on a pro forma basis, it would have yielded a higher rate of agreement between analysts- and algorithmically assigned Process pillar ratings. We also found that pro forma would have improved the efficacy of ratings assigned to passively managed equity vehicles.
Strategic beta funds will continue to be assessed under the current random forest model.
From 29 October 2024, all Morningstar Medalist Rating data updates will flow automatically into all products and feeds containing this data including Adviser Research Centre (ARC). However, you should be aware of this change, as certain funds may shift in their ratings, and you may need to adjust your saved items or searches, as necessary.
We feel confident these changes will enhance the efficacy of the Medalist Ratings based on extensive testing of the revised approach that we’ve conducted and would improve the Medalist Ratings’ overall predictiveness and efficacy.
We will continue to explore additional methodological enhancements in our effort to further improve the Medalist Ratings’ usability in your work assisting investors.
If you have any questions or feedback, please contact our support team.
- Phone: 1800 034 455 (AU), 0800 888 363 (NZ)
- Email: help.au@morningstar.com