Effect of State-Led Delivery System and Payment Reforms on Population-Level Detection and Management of Diabetes

Since 2013, the Centers for Medicare and Medicaid Services (CMS) State Innovation Models (SIM) initiative has been supporting states with financial resources and technical support to develop and test state-led multipayer health care payment and delivery reforms. The goal of SIM is to accelerate improvements in population health and decrease health care costs for all residents of grantee states. Using a competitive application process, the SIM initiative supported the testing and implementation of SIM plans in 17 states; in April 2013, six states received 42 months of test award funding totaling $254 million (round 1 states), and in January 2015, 11 states received 48 months of test award funding totaling $622 million (round 2 states) (1).

In order to achieve SIM goals, most grantee states used their funds to advance multiplayer alignment to test value-based payments and invest in reforms to improve detection and management of chronic conditions, particularly type 2 diabetes. Diabetes detection and management were a focus because of the large contribution of diabetes to patient morbidity and health care costs (2,3). In 2017, an estimated 24.6 million adults were diagnosed, incurring $236 billion in direct medical costs attributable to diabetes, of which hospital inpatient costs comprised ∼29% (3). In 2014, there were 7.2 million hospitalizations of adults with a diabetes diagnosis, a rate of 327 hospitalizations per 1,000 diagnosed adults (4). The written plans of all grantee states included outcome and use measures focused on adults with diabetes (5). The first year of CMS SIM initiative round 1 was associated with higher diagnosed diabetes prevalence among adults, but results related to lower hospitalization rates among adults diagnosed with diabetes were mixed (6). Recent evidence also indicates that the CMS SIM initiative had a population-level effect on self-rated physical and mental health among adults age ≥45 years (7). Past research also highlights that SIM implementation varied considerably across grantee states, including the extent to which states focused on improving behavioral health integration into primary care and achieved multipayer alignment to test value-based payment reforms. Information about variation in SIM implementation by state is detailed elsewhere (8–12).

In this study, we analyzed whether adults in SIM initiative round 1 or round 2 states experienced an increase in diagnosed diabetes prevalence or a decrease in hospitalization or readmission rates among those diagnosed, relative to patients in comparison states. SIM requires the alignment of multiple payers within each grantee state, which can improve population-level detection and management of diabetes by implementing interventions to identify and treat undiagnosed adults with diabetes and by improving the chronic care management capabilities of health care organizations. These efforts could result in increased diabetes detection and lower hospitalization rates and rates of readmission within 30 days of discharge among diagnosed adults.

Based on the evidence of the positive association of SIM with population-level physical and mental health status (7) and evidence of the positive effect of state-led health reforms on diabetes care management among low-income patients (13,14), we anticipated that the SIM initiative would increase the detection of diabetes within 2 to 4 years of implementation (hypothesis 1). We also expected that SIM participation would improve the management of diabetes, resulting in fewer hospitalizations among adults diagnosed with diabetes, with more pronounced effects on hospitalization resulting from ambulatory care sensitive conditions (ACSCs), because these encounters are considered preventable through primary care management of diabetes and other chronic conditions (15–17), which was a major focus of the CMS SIM initiative (hypothesis 2). Finally, we expected that the SIM focus on aligning incentives across organizations would improve post–acute care of hospitalized adults with diabetes, resulting in fewer hospital readmissions within 30 days of discharge (hypothesis 3).

Data on the number and prevalence of adults diagnosed with diabetes per county-year were from the Centers for Disease Control and Prevention County Data Indicators (18,19) for residents age ≥18 years. The Area Health Resources File was used to obtain data on county-level socioeconomic deprivation and demographics for county-level analyses.

Encounter-level hospitalization data were from the State Inpatient Databases (SIDs) of all-payer hospitalizations maintained by the Healthcare Cost and Utilization Project (HCUP). The six round 1 SIM states included were Arkansas, Maine, Massachusetts, Minnesota, Oregon, and Vermont. The six round 2 states included were Colorado, Iowa, Michigan, New York, Rhode Island, and Washington. The comparison group included five states: Arizona, Florida, Georgia, New Jersey, and New Mexico.

For analyses of 30-day readmissions, only nine of the 17 states were included, because the other eight HCUP SID files do not allow for the tracking of individuals over time. SIM states included in the readmission analyses were Arkansas, Massachusetts, and Vermont for round 1 and Iowa, New York, and Washington for round 2. The three comparison states for the readmission analyses were Florida, Georgia, and New Mexico.

Hospitalizations among adults diagnosed with diabetes were identified based on Agency for Healthcare Research and Quality specifications using HCUP Clinical Classifications Software (CCS), which categorizes diagnosis codes into clinical groupings (20). HCUP SIDs include a census of all hospitalizations across all payers. The included hospitalizations were those in which patients had any listed diagnosis of diabetes without complications (CCS category 49 for ICD-9, END002 for ICD-10) or diabetes with complications (CCS category 50 for ICD-9, END003 for ICD-10), which were extracted using principal and secondary diagnosis fields for each patient. This study was approved by the Office of Human Subjects Protection at the University of California, Berkeley.

Supplementary Table 1 details the analytic sample restrictions of the study. For analyses of hospitalizations, we analyzed a census of all hospitalizations in the 17 states comprising 889 counties from 2010 to 2017 (8 years) with 21,055,714 hospitalizations of adults age ≥18 years diagnosed with diabetes, ∼4.5 million (22%) of which were for ACSCs. The analytic sample for the hospital readmission analyses included 11,812,993 hospitalizations for adults with diabetes age ≥18 years from 2010 to 2017 across the subset of nine states, of which 7,730,451 were index admissions and 948,718 were readmissions within 30 days. Of index hospital admissions, 954,183 occurred in SIM round 1 states, 2,866,297 occurred in SIM round 2 states, and 3,909,971 occurred in comparison states.

The main independent variable for the event study design was defined at the county-year level indicating whether a county was in a state that received an SIM initiative test award in round 1 or 2 and whether the year was in the SIM implementation period (1). The main independent variable for admission-level analyses was defined at the admission-year level indicating whether a hospital admission was in an SIM state and whether the year was during the SIM implementation period of that state (1).

SIM round 1 funding began on 1 April 2013, which included an initial 6 month test period, resulting in the test implementation phase beginning 1 October 2013 (5). SIM round 2 funding began on 1 February 2015, which included an initial 6 month test period, resulting in the test implementation phase beginning 1 August 2015. The SIM implementation start dates used for the analyses were 1 January 2014 for round 1 states and 1 January 2016 for round 2 states, given the annualized nature of the study outcomes.

The outcome measures were as follows: 1) diagnosed diabetes prevalence among adults, 2) ACSC hospitalizations per 1,000 diagnosed adults, 3) all-cause hospitalizations per 1,000 diagnosed adults, and 4) all-cause 30 day hospital readmissions. The first three outcomes were measured at the county-year level for adults (age ≥18 years) from 2010 to 2017, because counties are the geographic units for the Behavioral Risk Factor Surveillance System diagnostic prevalence data and the most appropriate unit for analyses of all-cause hospitalization rates. The 30-day readmission outcome was measured at hospital admission level.

Adults were considered diagnosed with diabetes based on their response to the following Behavioral Risk Factor Surveillance System question: “Has a doctor ever told you that you have diabetes?” Women who had been diagnosed with gestational diabetes only were not considered diagnosed (18,19).

We identified hospitalizations resulting from ACSCs from ICD-9-CM and ICD-10-CM codes that were extracted from the principal diagnosis field of each patient using the following 2015 Agency for Healthcare Research and Quality Prevention Quality Indicators: 1) diabetes short-term complications, 3) diabetes long-term complications, 5) chronic obstructive pulmonary disease or asthma in older adults, 7) hypertension, 8) heart failure, 10) dehydration, 11) bacterial pneumonia, 12) urinary tract infection, 13) angina without procedure, 14) uncontrolled diabetes, 15) asthma in younger adults, and 16) lower-extremity amputation among patients with diabetes (21). Annual county-level hospitalization rates were calculated by dividing the number of hospitalizations by the number of adults (in 1,000s) diagnosed with diabetes.

For the 30-day readmission outcome, a patient’s first hospitalization in the calendar year was marked as an index admission to assess a readmission within 30 days.

Our regression approach differed for the study outcomes, because trends in SIM states versus comparison states for the study outcomes differed during the pre-SIM and SIM implementation periods. To estimate the association of SIM with diabetes prevalence and hospitalization rates, an event study design was used, because the pre-SIM trends of these outcomes were similar between SIM and comparison states. We conducted comparative interrupted time series (CITS) analyses to assess the association of SIM and 30-day readmissions, because visual assessment of trends in readmission rates in SIM states versus comparison states revealed that trends were not parallel in the pre-SIM period. In contrast to difference-in-differences (DiD) models, which assess whether treated and comparison groups have differential changes in outcome means in the postintervention period relative to the preintervention period, CITS models assess whether treated and comparison groups have differential changes in outcome trends in the postintervention period relative to the preintervention period. Specifically, CITS models investigate whether the treated group diverges from its preintervention trend to a greater or lesser extent than the comparison group diverges from its preintervention trend (22,23).

To test hypotheses 1 and 2, DiD event study models were estimated using Eq. 1.

In Eq. 1, c indexes counties and t indexes time for the following variables: Y_c,t is the dependent variable, which is the percentage of adults diagnosed with diabetes (hypothesis 1) or the number of hospitalizations per 1,000 diagnosed adults (hypothesis 2); SIM_c indicates whether county c was located in an SIM state; post_k,c,t is a vector of K variables (excluding the reference period k = −1), in which k is the number of years from the year that the state implemented SIM (e.g., for an SIM round 1 state implementing SIM in 2014, k = −4 in 2010, k = −3 in 2011, k = −2 in 2012, k = −1 in 2013, k = 0 in 2014, k = 1 in 2015, k = 2 in 2016, and k = 3 in 2017, and for an SIM round 2 state implementing SIM in 2016, k ranges from −6 to 1); year_t is a vector of T indicator variables, one for each year (excluding the reference year 2010, in which t = 1); C_c is a vector of county fixed effects to control for time-invariant outcome differences among the counties; X_c,t is a vector of time-varying confounders measured as means at the county population level, including sex, adult age distribution, race/ethnicity, uninsured rate, and poverty rate, which are known to affect diagnosed diabetes prevalence among adults and hospitalization of diagnosed adults. The equation does not include a SIM_c main effect, because it would be collinear with the county fixed effects. The parameters of interest are λ_k, and each DiD parameter λ_k estimates the DiD for each k relative to k = −1 (the reference period, used by convention).

We conducted index admission–level CITS analyses, because visual assessment of trends in 30-day readmission rates in SIM states versus comparison states indicated that trends were not parallel in the pre-SIM period. CITS models allow for nonparallel preperiod trends that are required of DiD models (24). Separate models were estimated for each SIM round, because there may have been heterogeneous effects between the two cohorts of SIM states. To test hypothesis 3, a CITS model was estimated using logistic regression with Eq. 2.

in which i indexes index hospitalizations and t indexes time for the following variables: Y_it indicates whether an index hospitalization was followed by a readmission within 30 days; Post_t indicates whether the discharge date of an index hospitalization occurred during SIM implementation; Time_t is a continuous yearly time trend for the study period, beginning with 1 and ending with 8 to represent 2010 to 2017; SIM_i indicates whether an index hospitalization occurred in an SIM state versus a comparison state; X_it is a vector of covariates that includes age, sex, race/ethnicity, length of stay, admission via the emergency department, comorbidity burden, and primary payer; and Hospital_i is a vector of hospital fixed effects to account for time-constant hospital-level variation in readmission rates. Therefore, SIM_i was not included by itself because of collinearity with these fixed effects.

The interpretation of the interaction terms is as follows: SIM_i × Post_t represents the difference in the outcome between SIM states and comparison states during SIM implementation; SIM_i × Time_t represents the difference in slope between SIM and comparison states before SIM implementation; Post_t × Time_t represents the change in slope in comparison states during SIM implementation; and the triple interaction of SIM_i × Post_t × Time_t is the parameter of interest, corresponding to the difference between SIM and comparison states in slope during SIM implementation compared with before. SEs were estimated by clustering at the state level to allow for correlation within hospitals across time.

To address potential confounding variables in the preintervention period that affected outcomes in the postintervention period, our models used inverse probability of treatment weights (IPTW) in the form of average treatment effect in the treated (25). This approach allowed for statistical assessment of whether comparison states were equivalent to SIM states for the confounding covariates after balancing. Propensity scores were generated by way of a multigroup weighting strategy, which used multinomial logistic regression to estimate the probability of each observation being in the treated group in the pretreatment period (26).

The following county-level variables were used to estimate the scores for the event study analyses, because they are related to diabetes prevalence and hospitalization rates: sex, age, race/ethnicity, uninsured rate, and poverty rate (income <100% of the federal poverty level). Other variables were also considered, but a parsimonious set of variables was required in order to achieve sufficient balance for the sample of 889 counties. For the CITS analyses, the following patient-level variables were used to estimate propensity scores: age, comorbidity count, and indicators for being female, Black, Hispanic, other racial/ethnic minority, having Medicaid, being a dual Medicare/Medicaid enrollee, having private insurance, being uninsured/self-paying, and being admitted through the emergency department.

The weight of each observation was then calculated to be proportional to its probability of being in the SIM group before SIM relative to its probability of being in the SIM group and the time period in which it truly occurred (i.e., SIM group during SIM, non-SIM group pre-SIM, or non-SIM group during SIM). Our goal was to have the preintervention period absolute standardized differences of the variable means between the SIM and comparison groups to be <10% (27,28). The IPTWs were used as probability weights to estimate the regressions, thus making them doubly robust (29). This approach was previously used to study the effect of Medicare hospice enrollment on costs and quality (30). The DiD regression models were weighted for a county population by multiplying the county population by the IPTW weights. SEs were estimated by clustering at the state level to allow for correlation within states across time.

We conducted two sensitivity analyses to examine the robustness of our results to alternative model specifications. First, we restricted the analytic samples to adults age ≥45 years, because SIM was previously found to improve physical and mental health status of adults at a population level (7). Second, we restricted the analytic samples to Medicare and/or Medicare beneficiaries age ≥18 years, because SIM is sponsored by CMS, and over time, effects may be different for CMS populations.

Table 1 compares the weighted outcome measures, county-level population measures, and admission-level socioeconomic and clinical characteristics for adults and for hospitalized adults diagnosed with diabetes in SIM versus comparison counties from 2010 to 2017. In SIM versus comparison counties, the crude diagnosed diabetes prevalence was lower (9.3% in both round 1 and round 2 counties vs. 10.2% in comparison counties), and the all-cause and ACSC hospitalization rates were also lower (272.6 in round 1 counties and 305.1 in round 2 counties vs. 320.7 in comparison counties and 58.4 in round 1 counties and 65.1 in round 2 counties vs. 73.1 in comparison counties, respectively). At the population level, SIM counties were demographically similar to comparison counties, but some differences were seen in the percentage of those who were White (80.8% vs. 74.1%) and uninsured (12.4% vs. 21.5%). Comparison counties had a pre-SIM trend that was significantly steeper than the SIM counties trends for ACSC hospitalization rate (1.9 hospitalizations per 1,000 diagnosed adults steeper decrease per year; P < 0.01) and for all-cause hospitalization rate (5.8 hospitalizations per 1,000 diagnosed adults steeper decrease per year; P < 0.01). The use of propensity score weights achieved better covariate balance compared with unweighted analyses, with standardized mean differences for variables between groups within acceptable ranges (standardized mean differences were all greater than five) (27,28) (Supplementary Fig. 1).

During the study period, the weighted mean diagnosed diabetes prevalence among adults increased in both SIM and comparison counties, but the prevalence level was lower in SIM counties (Supplementary Fig. 2). The regression-adjusted relative DiD parameters for SIM × time are plotted in Fig. 1, which shows that relative to the year before SIM implementation (k = −1), the relative outcome DiD did not significantly change after SIM was implemented (k ≥ 0). Diabetes prevalence is presented in panel A, all-cause hospitalization rate is presented in panel B, and rate of hospitalization because of ACSCs is presented in panel C.

The weighted mean all-cause and ACSC hospitalization rates per 1,000 adults diagnosed with diabetes decreased in both SIM and comparison counties during the study period (Supplementary Figs. 3 and 4). Although the relative DiDs started to increase after SIM was implemented (k ≥ 0), meaning more hospitalizations occurred in SIM states relative to comparison states, controlling for their difference in the period before SIM was implemented (k = −1), the relative DiDs were not statistically significant at the 0.05 level. Results of the full regression models are reported in Table 2.

For the readmission results, during the pre-SIM period, round 1 states experienced a decrease in their weighted readmission rates relative to comparison states, but during SIM implementation, the relative trends reversed, with SIM states experiencing an increase in 30-day readmission rates relative to comparison states (Supplementary Fig. 5). The regression-adjusted results corroborated these weighted trends, with the estimate for the parameter of SIM × post × year being 1.07 (95% CI 1.04, 1.11; P < 0.001) (Table 3). This interaction term reflects the change in slope in odds of readmission in SIM states after implementation compared with before relative to the change in slope in comparison states. Specifically, it indicates that the change in odds of 30-day readmission among hospitalized adults with diabetes in SIM states during implementation compared with before was 7% greater than the change in odds of readmission for adults with diabetes in comparison states over this time period.

The results for SIM round 2 states were similar. During the pre-SIM period, round 2 states also experienced a decrease in their weighted 30-day readmission rates relative to comparison states. During the post-SIM period, trends were more difficult to assess, because this period was only 2 years (2016 and 2017), but it is clear that comparison states experienced an increase in readmission rates as compared with round 2 states (Supplementary Fig. 6). Notwithstanding, because of the relative trend differences in the pre-SIM period, the regression-adjusted results report the estimate for the parameter of SIM × post × year to be 1.06 (95% CI 1.03, 1.10; P < 0.001) (Table 3), indicating that SIM states had a 6% greater change in odds of readmission for adults with diabetes compared with comparison states.

The DiD and CITS model results were consistent when the analytic sample was restricted to adults age ≥45 years (Supplementary Tables 2 and 4) and when restricted to Medicare and/or Medicaid beneficiaries (Supplementary Tables 3 and 5). To aid the interpretation of significant SIM effects, we reestimated the CITS regression models as linear probability models for the 30 day all-cause hospital readmission outcome measure (Supplementary Table 6), and the results were consistent with those of the logistic regression models.

State-led delivery system reforms that aim to improve population health through chronic care management like the CMS SIM initiative are increasingly common, because they have high potential to incentivize and accelerate improvements in population-level prevention and chronic care management through improved coordination of health care and public health organizations and systems. Despite this potential, our study results indicate that SIM has not yet had a measurable population-level effect on diabetes detection or management. For example, we found that the increased diabetes detection trend among SIM states documented in past research was only a temporary pattern. Although improving diabetes detection and management for adults with diabetes was a focus of SIM plans in most states because of the increasing prevalence of diabetes during the last three decades (3), our findings through 2017 indicate that SIM implementation did not significantly improve diabetes detection (hypothesis 1), reduce all-cause hospitalizations or hospitalizations resulting from ACSCs (hypothesis 2), or reduce all-cause 30-day readmissions (hypothesis 3) among adults with diabetes. These results were consistent when the analytic sample was restricted to CMS populations, highlighting that SIM was not associated with better diabetes detection or management among Medicare or Medicaid beneficiaries, who would be the most likely to benefit given the CMS sponsorship of the initiative.

One reason that SIM implementation may not have been associated with better diabetes detection or management is because grantee states emphasized different strategies to achieve their goals, including value-based payment models, accountable care organizations, health homes for individuals needing behavioral health services, and regional collaborations of medical and long-term service and support providers (10). Moreover, a key challenge for SIM states was achieving multipayer alignment to sufficiently test and scale successful delivery system and payment reforms (8–12). Investments in health information technology and chronic care management processes, while helpful to populations with diabetes, may have been too diffuse and limited in scale to increase diabetes detection or improve diabetes care management. This explanation for our study findings is consistent with recent research that found that SIM was not associated with greater physician practice–level improvements in chronic care management or health information technology capabilities compared with non-SIM states (31).

The results related to the association between SIM and 30-day readmissions are consistent with past evidence through 2015 indicating that SIM implementation is associated with increased 30-day readmissions among adults with diabetes (32). In updated analyses through 2017, we also found that SIM states had greater increases in odds of 30-day readmission after SIM implementation compared with comparison states. Importantly, these effects were consistent for round 1 and round 2 grantee states. It may be that these efforts to align interests and coordinate with payers distract from the internal performance improvement efforts of health care organization focused on improving care transitions, because excessive 30-day readmissions were penalized during this period by the CMS Hospital Readmissions Reduction Program, which decreased 30-day readmissions nationally (33–36).

This study has limitations that warrant discussion. First, our methods assumed the absence of interventions that occurred contemporaneously with SIM that affected the study outcomes. Our models controlled for the uninsured rate of each county, which controls for Affordable Care Act–related expansions that differ by state. However, since launching SIM, several federal initiatives have been implemented, including the CDC National Diabetes Prevention Program and CMS efforts to improve care for Medicare and Medicaid dually eligible patients (37). These initiatives may have been implemented differentially between SIM and comparison states, but it was not feasible to measure or incorporate data about these initiatives into our analyses. Second, few non-SIM states participate in the HCUP SIDs, so comparison states were limited to the few non-SIM states with consistent HCUP SID data. Third, we conducted county-level analyses, which may have been subject to ecologic bias and constrained our ability to include multiple contextual variables, including measures of the food environment. Relatedly, we could not consider information about heterogeneity of SIM implementation by state because of the limited number of SIM states with HCUP SID data and the high variation across states (8). Future research should examine whether SIM states with a strong emphasis on behavioral health integration into primary care have improved diabetes management (9).

In conclusion, our results highlight that broad-based state-led delivery system and payment reforms, although bold and potentially transformational in improving physical and mental health status at the population level, are unlikely to affect diabetes detection or management after 2 to 4 years of implementation. In the past, delivery and payment reforms have modestly reduced health care spending, including hospital expenditures, among early provider participants (38,39), likely because they had more advanced systems to manage care. State-led reform efforts, which require extensive cooperation of multiple payers and health care organizations, may be challenged with improving diabetes detection and management because of the challenges of stakeholder alignment that have limited the testing and dissemination of SIM system reforms (8–12) and the resulting mixed effects on diabetes detection and care management. The results of this natural experiment of SIM implementation underscore the importance of managing stakeholder expectations about the short-run effect of broad-based state-led delivery systems and payment reforms on improved diabetes care.

Acknowledgments. The authors are thankful for critical feedback from collaborators in the Natural Experiments in Translation for Diabetes Study 2 (NEXT-D2) network and from our project advisory panel of State Innovation Models Initiative directors and staff. The authors also thank Salma Bibi for the lead role with project management.

Funding. This publication was made possible by NEXT-D2, a cooperative agreement (5U18DP006123) jointly funded by the National Center for Chronic Disease Prevention and Health Promotion of the Centers for Disease Control and Prevention and the National Institute of Diabetes and Digestive and Kidney Disease, National Institutes of Health.

Duality of Interest. No potential conflicts of interest relevant to this article were reported.

Author Contributions. H.P.R. was responsible for conceptualization, methodology, investigation, original draft preparation, reviewing and editing of the manuscript, funding acquisition, and supervision. B.D.F. was responsible for methodology, investigation, reviewing and editing of the manuscript, and supervision. A.Z.P. was responsible for methodology, investigation, visualization, and reviewing and editing of the manuscript. K.R. was responsible for investigation, visualization, and reviewing and editing of the manuscript. H.P.R. and B.D.F. are the guarantors of this work and, as such, had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.