Medicare Advantage Star ratings may decline with new methodology

The Centers for Medicare & Medicaid Services (CMS) uses a Star rating system to measure the performance of Medicare Advantage (MA) plans. MA plans overall saw an unprecedented improvement in Star performance in rating year 2022¹Published every year, the CMS Five-Star Quality Rating System measures the quality of health and drug services. Years refer to the rating year rather than the measurement year, unless otherwise noted.: the enrollment-weighted average contract rating rose to 4.37 Stars, an all-time high in the program’s 12-year history.²“Fact sheet - 2022 Part C and D Star ratings,” CMS, October 8, 2021. This performance improvement was driven by temporary, favorable changes to the Star rating methodology initiated by CMS in response to the COVID-19 pandemic and ongoing investments by plans in quality improvement initiatives (see sidebars, “How ratings cycles work” and “Medicare Advantage Star ratings rose to all-time highs in 2022”).

This level of performance is unlikely to continue, however, as three major changes to the methodology that CMS uses to calculate Star ratings go into effect.³“CY 2023 Medicare Advantage and Part D final rule (CMS-4192-F),” CMS, April 29, 2022. First, for rating year 2023, CMS will no longer universally apply the disaster provision as it did in 2022; this provision allowed contracts to take the “better of” current or historical performance for most measures in 2022.⁴“Fact sheet,” October 8, 2021. Second, CMS will implement annual upper and lower limits, or guardrails, on changes in cut points for non-CAHPS⁵Consumer Assessment of Healthcare Providers and Systems (CAHPS), a survey that CMS uses to assess patient experience of care. measures beginning in rating year 2023.⁶“Fact sheet,” October 8, 2021. Cut points are the ranges that a contract’s score on a particular measure needs to fall within to achieve each Star value. Third, CMS will remove performance outliers from the calculation of non-CAHPS measure rating cut points in rating year 2024⁷“Contract year 2021 Medicare Advantage and part D final rule (CMS-4190-F1) fact sheet,” CMS, May 22, 2020. using the Tukey outlier deletion method.⁸Tukey outlier deletion is a statistical method for identifying and removing outliers based on the interquartile range of the set. Data points that are more than 150 percent of the interquartile range above the third quartile are identified as outliers and removed, as are points less than 150 percent of the interquartile range below the first quartile.

MA plans that are unable to maintain their current Star rating performance under the new cut point methodologies or that cannot turn around performance that declined during the pandemic may face adverse financial impacts resulting from reduced Medicare bonuses and rebates. Our analysis suggests that half of plans could see reduced ratings in rating year 2023 based on changes to the disaster provision, and that more challenging cut points could separately drive $800 million in annual revenue impact to plans in rating year 2024. To preempt negative impact, plans can consider conducting diagnostic analyses to identify at-risk measures and deploying targeted initiatives to address those measures. In the longer term, cut points will become more predictable, enabling plans to invest more confidently in quality improvement initiatives. This article describes the forthcoming changes to the Star ratings methodology, implications for MA plans, and actions that plans can consider to be ready for the changes.

Upcoming changes to the Star ratings methodology and the implications

The changes to CMS’s methodology for calculating Star ratings will have varying effects on plans, largely depending on the measure category and contract-level performance.

The limited application of the ‘better of’ methodology could reduce ratings for half of contracts

In response to the COVID-19 pandemic, CMS allowed all contracts to use the existing disaster provision for rating year 2022, causing Star ratings to be calculated based on the “better of” current or historical performance for most measures.⁹“Fact sheet,” October 8, 2021. This provision was used for an estimated 21 percent of measure ratings nationally (Exhibit 1). Rating increases from rating year 2021 to 2022 were 30 to 40 percent higher than the average seen in rating years 2017 to 2021 because the disaster provision enabled contracts to benefit from performance improvements without factoring in performance deterioration. Likewise, rating decreases from rating year 2021 to 2022 were were 15 to 25 percent lower. In rating year 2023, when this “better of” methodology no longer applies universally, average contract Star ratings may revert from the average of 4.37 across all contracts in rating year 2022 to 4.07–4.17, their levels during rating years 2019–21, before the methodology was expanded to accommodate the pandemic. They could even go lower if underlying performance has deteriorated since rating year 2021.¹⁰CMS Star Ratings data (2013 to 2022) and September enrollment data (2013 to 2021). Note that enrollment weighted Star ratings may differ slightly from CMS reports due to enrollment data used in analysis.

Contracts used “better of” performance most frequently for CAHPS measures, a category that has been gradually increasing in weighting since 2020¹¹“New Stars ratings,” October 15, 2020. and will reach its target weight in 2023. Thirty percent of weighted CAHPS measures received equivalent measure scores and ratings in rating years 2021 and 2022, a sign that plans may have used historical rather than current-measure performance. Moreover, 78 percent of contracts saw decreased ratings in at least one CAHPS measure from 2019 to 2020, the last year before COVID-19 began affecting CAHPS measurement. Our analysis suggests that deteriorating CAHPS performance could have a significant impact on MA plans’ overall Stars performance.

Guardrails will increase cut point stability

CMS’s introduction of guardrails limiting year-over-year changes in cut points will make cut points more stable beginning in rating year 2023. This change will enable plans to more accurately predict the level of performance required on each measure to achieve a certain Star rating.¹²“Fact sheet,” October 8, 2021.

The average change in cut points for 4-Star ratings was 5 percent from 2019 to 2020, with individual cut points changing by as much as 10 percent (Exhibit 2). If guardrails had been in place during that same period, the average change would have been 3 percent for 4-Star ratings; individual measure changes would have been limited to 5 percent.¹³Guardrails will limit the year-over-year change of measure cut points to five percentage points for measures with a zero to 100 scale, or 5 percent of the restricted range for other measures. The effect of guardrails would have been even more material for plans with 2- and 3-Star ratings.

Overall, cut points will change less from year to year across all performance levels. This will make those cut points more predictable, increasing plans’ visibility into the outcomes that will be required from quality initiatives to maintain or improve Star ratings.

Tukey outlier deletion will make it more challenging for plans to maintain or improve Star ratings

By eliminating performance outliers from the calculation of Star rating cut points using the Tukey methodology beginning in rating year 2024, CMS will make it more challenging for plans to achieve the performance required to maintain or improve Star ratings. This is because outliers are more commonly found at the low end of performance among poorly performing contracts. Our analysis suggests that MA plans could see an $800 million annual revenue impact from more challenging cut points.

A retroactive analysis of the effects of cut point changes

To understand the potential effect of cut point methodology changes on MA contract Star performance, we retroactively analyzed the impact they would have had on performance in rating year 2020. (We chose 2020 because temporary methodology adjustments related to COVID-19 had not yet gone into effect.) We analyzed all 401 national MA prescription drug (MA-PD) contracts rated in 2020. Sixty-nine of these contracts—17 percent of contracts, enrolling 9 percent of beneficiaries—would have achieved a 0.5 lower Star rating in rating year 2020 (for example, achieving 3.5 Stars rather than 4.0 Stars) had Tukey and cut point guardrails been in place (Exhibit 3). No contracts would have seen higher Star ratings. These 69 contracts would have lost an average of $28 per member per month in performance-related bonus and rebate revenue because of the rating change, equating to $800 million in annual revenue impact across all contracts.

Across all 401 contracts, cut point changes would have decreased average ratings by 0.1 Stars. While our results do not predict future trends in raw-measure performance, they illustrate that changes to the methodology for translating raw performance into Star ratings may create financial challenges for health plans in future years.

Contracts with 3.5- and 4.0-Star ratings would have been most affected

In our retroactive analysis, the contracts most affected by cut point changes were those that received 3.5- and 4.0-Star ratings in rating year 2020; 23 percent of these contracts (enrolling 14 percent of beneficiaries) would have had been rated 0.5 Stars lower (Exhibit 4). Although cut points in this performance range were less affected than those in lower ranges (1.0, 2.0, and 3.0 Stars), 3.5- and 4.0-Star contracts tended to sit closer to the overall rating thresholds, reducing the buffer for these contracts to absorb even small changes in measure performance. MA contracts with 4.0-Star ratings would be particularly affected financially by these changes due to the loss of the 5 percent quality bonus (which 3.5-Star contracts are not eligible for).

Changes to even a few measures can affect a contract’s Star rating

Not all measures would have been affected by the outlier removal and guardrails. In fact, the impact was heavily concentrated on a small number of measures, illustrating that changes to the cut points of even a few measures can have a significant impact on Star ratings.

The “Complaints about the health/drug plan” measure would have been most affected in 2020 by cut point methodology changes, with 98 percent of contracts seeing performance on that measure fall by at least 0.5 Stars when changes were implemented (Exhibit 5). The cut points for this measure, in which a lower score corresponds to stronger performance, increased substantially from rating year 2019 to rating year 2020 as some plans struggled with elevated complaint volumes. If outlier removal and guardrail methodologies had been in place, the cut point increases would have been much smaller, making it more difficult for contracts to achieve higher Star ratings. For example, a contract that registered 0.32 complaints per 1,000 members earned a 5-Star rating on this measure in 2020, but it would have earned a 3-Star rating under the new cut point methodology.

The bottom line for plans

Many MA plans that relied on the “better of” methodology in rating year 2022 may see lower Star ratings in 2023. The financial impact may be substantial, particularly for those that have struggled with declining quality performances during the pandemic.

While the tendency for cut points to become more challenging each year is not new, plans should expect the increase in difficulty to accelerate in the short term. The new Tukey outlier removal methodology will mean that poorly performing plans will no longer bolster the ratings of other plans.

In the longer term, MA plans will no longer need to guess which measures will have significant fluctuations in cut points, because those fluctuations will be constrained by new guardrails beginning in rating year 2023. The return on investment in non-CAHPS measure improvements may increase because of this stabilizing methodology. Investments in CAHPS measures will remain important, as these measures are excluded from the guardrail provision and will increase to a weight of four beginning in rating year 2023.

Three actions for MA plans to consider

The good news for MA plans is that there is still time to act before the measurement period for rating year 2024 closes in December 2022. Specifically, plans can consider three types of actions:

• Plans could conduct diagnostic analyses to identify underlying performance challenges that prompted the plan to use the “better of” methodology in 2022. While some challenges may have been temporary as plans adjusted to the pandemic, others may be foundational and could put future Star performance at risk. For example, plans that are struggling with elevated complaint volumes could analyze the root causes of those complaints and launch targeted initiatives such as member outreach campaigns to avert poor member experiences before they occur.
• Plans could deploy targeted initiatives to address at-risk measures. Even for measures for which the measurement period is already under way, deploying a “SWAT team” targeted at specific measures could substantially improve Star performance. For example, plans have successfully launched midyear initiatives targeting Healthcare Effectiveness Data and Information Set (for example, call campaigns promoting statin use), pharmacy measures (for example, incentive programs with retail pharmacies), and operational measures (for example, mystery-shopper campaigns to assess call-center performance).
• Plans could consider developing a programmatic approach to quality investments or increasing levels of investment if they already have such an approach. The ROI of these investments may increase as cut points become both more challenging and less volatile in future years. For example, leading plans have delivered quality-focused training to front-office staff, deployed interactive text messaging, and used predictive analytics to launch call campaigns focused on medication adherence.

---

The next several years may be difficult for MA plans as CMS shifts to a more demanding Star rating methodology that will require contracts to perform better to maintain current ratings. For many plans, particularly those that have yet to regain footing in their quality programs during the pandemic, the financial impact will be substantial.

However, for plans that successfully adapt to a more demanding Star ratings environment, quality may emerge as a source of competitive advantage. Maintaining a Star rating of four or higher can help plans remain financially stable, offer rich supplemental benefits for members, and compete for the 56 percent of enrollees¹⁴Marina Ivanenko, Dan Jamieson, and Cara Repasky, “What Medicare Advantage members want from their onboarding experience,” McKinsey, April 18, 2022. for whom Star ratings are a top buying factor. Plans that lead on quality will have substantial advantages to better serve their members.