A Multi-Method Study of Patient Reach and Attendance in a Pragmatic Trial of Diabetes Shared Medical Appointments Free

The Invested in Diabetes study was a cluster-randomized, pragmatic, comparative effectiveness trial of two models of diabetes SMAs (26). Twenty-two primary care practices delivered either standardized or patient-driven diabetes SMAs to adults with type 2 diabetes. The practices consisted of 12 FQHCs and 10 clinics that were not FQHCs (i.e., practices in which patients typically have commercial insurance) located in Colorado and Kansas. The SMA conditions were assigned at the practice level using covariate-constrained randomization (27). Both conditions used the Targeted Training in Illness Management (TTIM) curriculum (28) to deliver 6 sessions of DSMES. TTIM session topics included 1) Introduction to Diabetes, 2) Diabetes Basics, 3) Problem Solving & Talking to Your Doctor, 4) Stress & Coping (with optional content specific to mental illness), 5) Nutrition, and 6) Physical Activity & Sleep.

Standardized SMAs included TTIM delivery by a health educator with a set topic order and an individual visit with a medical provider. Patient-driven SMAs included TTIM delivery by a multidisciplinary care team (i.e., a health educator, behavioral health provider, and peer mentor [trained layperson with lived experience of diabetes]), which aimed to address patient preferences for a multidisciplinary diabetes care team and peer support (29), along with the individual provider visit. During the first session of patient-driven SMAs, patients selected the order and emphasis of the remaining topics. The distinguishing differences between the standardized and patient-driven SMAs—a multidisciplinary care team with peer support and patient-driven content in the latter but not the former—are the elements thought to represent a more patient-centered approach and were hypothesized to help overcome barriers to participation (26,29). The full study protocol and subsequent protocol adaptations were published previously (26,30).

All diabetes SMAs were delivered in person from 2019 until the coronavirus disease 2019 (COVID-19) pandemic began in March 2020. Afterward, practices could transition to virtual delivery using phone- and/or video-conferencing. Practices eventually resumed using in-person delivery or could opt to continue with virtual delivery through the end of the active study period in December 2021. The Colorado Multiple Institutional Review Board (IRB) approved the project as expedited human subjects research. The IRB waived the requirement for patient consent to participate in diabetes SMAs because care was delivered using an evidence-based intervention in usual care settings, and outcomes data were de-identified except for service dates (i.e., the study used a Health Insurance Portability and Accountability Act–defined limited dataset). Patients who participated in qualitative interviews gave verbal consent.

Practices agreed to recruit either 36 or 72 patients with type 2 diabetes (depending on practice size and available resources) to participate in diabetes SMAs. To enhance generalizability and ensure consistency with our pragmatic trial design (26,31), there were no study-specified eligibility criteria for patients beyond being an adult with type 2 diabetes, not being pregnant, and being willing and able to participate.

Practices received assistance from a study-assigned facilitator to assist with planning patient recruitment and retention strategies but were ultimately responsible for patient accrual. Existing practice personnel invited patients to participate by various methods (e.g., calling lists of patients known to have type 2 diabetes, facilitating provider referrals, or posting flyers in the practice). Practices could remind patients of upcoming sessions through phone calls, text messages, and health portal messages. The study did not provide incentives to patients to encourage participation, although practices could provide monetary or nonmonetary incentives (e.g., cookbooks or food provided at sessions) to do so. Practice facilitators advised practices on how to bill patient insurance for diabetes SMAs, although practices made their own decisions about whether to seek reimbursement.

TTIM materials were provided to practices in both English and Spanish. Trainings were also conducted in English and Spanish.

Outcomes and measures were based on the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) component of the PRISM (Practical Robust Implementation and Sustainability Model) planning and evaluation framework (32,33). To assess reach (percentage and representativeness of patients who participated and barriers, facilitators, and reasons for participation) and attendance (number and types of SMA sessions attended), practices tracked patient participation in diabetes SMAs in a structured tracking spreadsheet. In the same spreadsheet, practices also tracked patient demographics, including age, sex, race, ethnicity, and preferred language, and SMA session details, including dates (used to assess delivery before or after the onset of the COVID-19 pandemic), modality (virtual or in-person delivery), and the selected topic order (for the patient-driven condition only). Practice staff recorded and de-identified data before transfer to the study team.

Additional practice characteristics were collected at baseline, including the number of patients with type 2 diabetes, and have been reported in detail previously (30). Measures included 1) the number and percentage of all patients known to have type 2 diabetes in each practice participating in diabetes SMAs (i.e., reach) and representativeness of participants relative to population characteristics, 2) the number of participants per diabetes SMA cohort, and measures of attendance, including 3) the number of sessions attended, 4) the percentage of patients attending each of sessions 1–6, and 5) the percentage of patients attending each session by topic (e.g., Nutrition).

To assess patient-reported reasons for participating in SMAs and barriers to and facilitators of participation, practices were asked to identify three to five people whom they had invited to participate in diabetes SMAs to also participate in a qualitative interview. Although the goal was also to include patients who had been invited to participate in SMAs but elected not to enroll, no such patients agreed to be interviewed. Interview participants were not required to have attended all six diabetes SMA sessions, but their cohort’s program must have been completed before they participated in the interview.

Study staff were given contact information of interested patients who had given permission and recruited patients for the interviews by phone or e-mail. Interview participants were compensated with a $50 gift card. The interviews used a semi-structured guide that included questions about how diabetes SMAs were offered to them, any monetary costs or inconveniences of participating, their perceptions of the SMA facilitators, and their overall evaluation of diabetes SMAs, as well as their perceptions of virtual delivery of SMAs, if applicable. Interviews lasted up to 60 minutes and were conducted in person or by phone, recorded, and professionally transcribed.

We report representativeness of participants in diabetes SMAs in reference to all adults living with diabetes in Colorado. Colorado data (34) were used for reference because all but two participating practices were located in Colorado. Differences in demographics and participation by study condition (standardized vs. patient-driven SMAs) and by practice type (FQHC vs. non-FQHC) were assessed using t tests and χ² analyses. Differences in attendance were evaluated by demographics, cohort size, timing (before vs. after the onset of the COVID-19 pandemic), and session modality (in-person vs. virtual delivery) using t tests and χ² analyses. Multiple linear regression was used to model differences in attendance by study condition and practice type after adjusting for demographics (age, sex, race, ethnicity, and preferred language) and cohort size, timing, and modality. ANOVA was used to analyze the effects of cohort size (1–9, 10–19, or ≥20 participants) on attendance. Analyses were conducted in SPSS and Microsoft Excel software.

To qualitatively evaluate reach and attendance (35), a team of three experienced qualitative researchers coded and reviewed interview transcripts in Atlas.ti, v. 9, software using a mix of deductive coding (based on the interview guide and overall research questions) and inductive coding. The team coded ∼20% of transcripts together to ensure consistent use of codes before independently coding the remaining transcripts. After an initial review of qualitative reach data, quotation reports from codes focusing on the patients’ perspective of recruitment and retention were produced. The team analyzed these reports using thematic analysis to help explain observed participation trends (36).

Demographics and reach outcomes are shown in Table 1 for the 1,085 patients who attended diabetes SMAs. Thirty-eight percent were adults ≥65 years of age compared with 45.8% of all adults living with diabetes in Colorado. Regarding sex, 59.0% of participants in diabetes SMAs were women compared with 50.3% of adults with diabetes in Colorado. Regarding race and ethnicity, 27.4% of participants in diabetes SMAs were Latinx or Hispanic compared with 27.5% of adults with diabetes in Colorado. Fifty-six percent of participants in diabetes SMAs were White (not Latinx or Hispanic) compared with 61.7% of adults with diabetes in Colorado, and 7.6% were Black (not Latinx or Hispanic) compared with 5.5% of adults with diabetes in Colorado. Practices offering patient-driven SMAs recruited 508 patients, and those offering standardized SMAs recruited 577 patients. Comparing demographics between conditions, patient-driven SMAs included significantly more men (44.9 vs. 37.6%), individuals in racial and ethnic minority groups (43.3 vs. 41.7%), and Spanish-speakers (24.4 vs. 9.2%) than standardized SMAs (all P < 0.05). FQHCs reached 582 patients in diabetes SMAs, and non-FQHC clinics reached 503 patients. In comparison with non-FQHC clinics, FQHCs tended to reach more women (63.1 vs. 54.3%), younger participants ≤65 years of age (79.6 vs. 41.0%), individuals in racial and ethnic minority groups (69.8 vs. 10.1%), and Spanish-speakers (30.4 vs. 0.2%) (all P < 0.05).

As shown in Table 1, participants attended an average of 4.0 ± 1.8 of 6 sessions overall. English-speaking and non-Latinx or Hispanic White participants attended significantly more sessions than individuals in other groups. English-speakers attended 4.1 ± 1.8 sessions on average, compared with 3.4 ± 1.7 sessions for Spanish-speakers (P < 0.001). Non-Latinx or Hispanic White participants attended 4.1 ± 1.7 sessions on average, compared with 3.7 ± 1.8 sessions for participants in other racial and ethnic groups (P < 0.001). In unadjusted models, there were no differences in session attendance between conditions or practice types. However, in covariate-adjusted models, greater attendance was observed in FQHCs (4.1 ± 0.1 [standard error] sessions) than non-FQHC clinics (3.8 ± 0.1 sessions) (P = 0.048). In FQHCs, greater attendance was also observed with virtual delivery of SMAs (4.5 ± 1.6 sessions) compared with 3.7 ± 1.8 sessions attended with in-person delivery (P < 0.001). In contrast, attendance at non-FQHC clinics was higher with in-person than virtually delivered SMAs (4.1 ± 1.7 vs. 3.8 ± 2.0 sessions attended, P = 0.043).

The mean cohort size was 7.5 ± 4.2 participants overall, close to the recommended 8–10 participants according to the study protocol. Attendance varied by level of cohort size (F[2,1066] = 3.78, P = 0.023). Participants in cohorts with 1–9 group members (n = 653) attended 4.1 ± 0.1 sessions, those in cohorts with 10–19 group members (n = 344) attended 3.7 ± 0.1 sessions, and participants in cohorts with ≥20 group members (n = 72) attended 3.9 ± 0.2 sessions on average. Analysis of contrasts indicated that there were only statistically significant differences between the cohorts with 1–9 participants versus those with 10–19 participants (t[1,1066] = −2.75, P = 0.006).

On average, the frequency of attendance was highest at the first session (80.9%), followed by a relatively steep decline by the fourth session (57.6%). Figure 1 shows the frequency of attendance at sessions 1–6. Attendance patterns were similar across conditions and practice types, except that patient-driven SMAs had a higher frequency of attendance at session 2 (78.5 vs. 71.8%, P = 0.011). Figure 2 shows the frequency of attendance for each session topic. For patient-driven SMAs in which patients could select the topic order and emphasis (as opposed to having a set topic order as in standardized SMAs), a relative gain in attendance was observed for the session on nutrition (70.7 vs. 61.2%, P = 0.001). In comparison, lower attendance was observed for the session that jointly covered problem-solving and talking to your doctor (57.4 vs. 69.1%, P < 0.001). Patient-driven cohorts most often chose nutrition as their first selected topic (44.8% of cohorts selected the nutrition session to be presented first) and most often chose problem-solving and talking to your doctor to be last (41.7% of cohorts selected this session to be presented last). There were no significant differences in attendance for each session topic between FQHCs and non-FQHC clinics.

Forty-three patients completed interviews. Of those, 33 were women and 10 were men; 40 interviews were conducted in English versus 3 in Spanish. Twenty-five interviewees went to a practice delivering standardized SMAs compared with 18 who went to a practice delivering patient-driven SMAs; 21 went to an FQHC compared with 22 who went to a non-FQHC practice.

Qualitative data were grouped into three categories, including 1) reasons for initially deciding to attend diabetes SMAs, 2) reasons to continue attending diabetes SMAs, and 3) barriers to and facilitators of engaging in virtual diabetes SMAs. Table 2 provides a summary of key themes, along with supporting quotes.

Receiving a recommendation from a trusted provider or care team member was motivating for patients. This motivation was attributed to inherent trust in a provider’s recommendation, because of the desire to please the provider, or because it felt good that someone showed care and wanted to help. Many patients said they learned about diabetes SMAs from their primary care provider, who recommended attending to improve diabetes control or other related goals. A common successful recruitment strategy was for patients to receive a call from someone from the clinic, such as a nurse or care coordinator, inviting them to join a diabetes SMA. Patients reported that this was motivating even if the invitation came from a clinic team member whom they did not already know. Being encouraged to attend diabetes SMAs by a spouse or other family member was also motivating for some patients.

Seeking general education and specific information about diabetes in a class format was another common reason to participate. Many participants noted a strong interest in nutrition content, whereas others were interested in education on medication, managing stress, being active, checking blood glucose, using an insulin pump, and general self-care. Factors such as wanting a refresher on education received in the past (sometimes decades ago), expecting to learn something new, having a group format, and learning information that would not easily be found online also motivated attendance.

Related reasons to participate included seeking improved diabetes control and overall health. Several patients noted that recent blood tests showed high glucose levels, prompting them to worry about their diabetes. Other patients were experiencing symptoms that led them to believe their diabetes was worsening. Diabetes SMAs were seen as an opportunity to help get their diabetes under control. Many patients noted that these test results and symptoms were signs that they could not manage diabetes on their own and that it was time to take diabetes control more seriously. Patients viewed diabetes SMAs as a way to be more proactive and to be held accountable while also gaining practical skills for diabetes management.

Other factors in deciding to participate included that diabetes SMAs were convenient and offered at no cost (i.e., for Medicaid beneficiaries or in practices that opted not to bill for services). Patients perceived diabetes SMAs as convenient when the SMA schedule corresponded with their availability or they could participate virtually by phone. Several patients mentioned that they were unable to participate when diabetes SMAs were first offered because of competing demands at the scheduled session time (e.g., Mondays at 9:00 a.m.). However, new dates and times worked for them when they were offered the next time (e.g., Thursdays at 2:00 p.m.). Of note, participants in patient-driven SMAs did not mention being told in advance about having a multidisciplinary team (including a behavioral health professional and peer mentor) or that they would have the opportunity to select the topic order. That is, participants appeared to be unaware of the distinguishing features between the patient-driven and standardized SMAs when signing up; thus, the added patient-centered features were not noted as motivators for attendance.

Several patients said they committed to regular attendance once they came to the first session and were motivated by gaining valuable information, especially as they heard more details on what topics would be covered and received program materials. Although some patients said they received detailed information in advance about what diabetes SMAs would cover, many said they received little information ahead of time beyond the general emphasis on diabetes education, as well as the schedule, format, and cost.

Receiving social support and individual attention also positively reinforced and encouraged their continued attendance. Several patients also mentioned that they valued socializing among peers, hearing from others experiencing similar struggles and challenges, and sharing ideas and stories. Having multiple sessions with the same cohort allowed more reserved or shy individuals to become comfortable with sharing. With patient-driven SMAs, the multidisciplinary care team was also seen as a benefit offering additional support, notably from the behavioral health professional.

Patient preference for virtual diabetes SMAs was more varied. Overall, some participants preferred virtual SMAs, whereas others would have preferred to meet in person, depending on their circumstances.

Barriers to engaging in virtual diabetes SMAs included perceiving a less enriching and engaging experience than in-person SMAs would have been and challenges with technology. One noted that participants could feel less accountable to the group and facilitator, especially if they joined without video and could therefore remain more anonymous. Several participants also found it harder to focus on the session content amid distractions from other participants (e.g., pets in the background), their own tendency to multitask, or fatigue from “sitting in front of a computer for 2 hours.” Technology issues were sometimes disruptive, including individual participants having difficulty connecting to the class and the entire platform malfunctioning for the class. Nonetheless, technology issues reportedly improved over time.

Facilitators of engaging in virtual diabetes SMAs included their safety and convenience, and participants’ perceptions that they were less emotionally taxing than in-person visits would have been. Regarding their convenience, virtual SMAs were perceived as easier to schedule around work and other responsibilities and less time-consuming overall because they did not require transit time to and from sessions. Participants also appreciated that virtual SMAs removed barriers such as lack of childcare or transportation, and virtual SMAs were preferable for patients who lived far from the clinic. In some cases, patients who were previously unable to attend in-person SMAs could later attend virtual SMAs because of their convenience. A subset of participants appreciated that virtual SMAs felt more private and less stigmatizing. This perception was especially important to participants who experienced mental health concerns and felt more comfortable in a virtual environment (without video, if they preferred). Overall, some participants reported that a virtual environment was more conducive to discussing sensitive topics. Nonetheless, participants also continued receiving social support from virtual SMAs, with some reporting that the level of support felt equivalent to meeting in person. Most participants also noted being readily at ease with the technology for virtual SMAs after initial training.

This report presents findings from a multimethod evaluation of patient reach and attendance in diabetes SMAs using the RE-AIM framework. Novel features of this study include evaluating two different models of diabetes SMAs (patient-driven and standardized SMAs) and examining the reach of diabetes SMAs in two types of primary care settings (FQHCs and non-FQHC clinics).

Diabetes SMAs reached an average of 50 patients per practice, corresponding to 6.2% of the patient population with diabetes across participating practices. In both the patient-driven and standardized models of diabetes SMAs, and in both FQHCs and non-FQHC clinics, overall attendance appeared equally high, with patients attending about 4 of 6 sessions on average. This level of attendance is consistent with recent studies of diabetes SMAs, even when different delivery models, settings, and patient populations were included (12,17,37). For example, 4 of 6 sessions were attended on average in diabetes SMAs that were enhanced with individual motivational interviewing sessions in VA settings (17). Four of six sessions were also attended on average in diabetes SMAs that provided the Conversation Maps curriculum to mostly Medicaid or uninsured patients in FQHCs (12). These findings suggest that practices may have leeway to use their preferred model of diabetes SMAs based on their available resources, priorities, setting, and patient needs.

Other novel features of this study included examining attendance at sessions 1–6 and for each session topic. Greater attendance was observed for the second session of the patient-driven versus the standardized models, which may reflect a greater interest in the nutrition content that was most often requested at this time (i.e., patients in patient-driven SMAs often chose nutrition content as the topic for second session rather than having it presented as the fifth session topic in standardized SMAs).

Regarding strategies to address health equity, delivering diabetes SMAs in FQHCs appeared to be key to reaching a high proportion of participants from racial and ethnic minority groups. This is important because individuals in racial and ethnic minority groups have higher rates of diabetes and experience diabetes onset at younger ages than White individuals (38). Notably, FQHCs reached five times more racial and ethnic minority participants and twice as many participants <65 years of age compared with non-FQHC clinics. Furthermore, greater attendance was observed with virtual delivery in FQHCs compared with in-person delivery of diabetes SMAs, suggesting that virtual delivery may help meet the needs of diverse and underserved patients who otherwise may have difficulty staying engaged in SMAs.

Another contribution of this study is its use of qualitative data to understand patients’ perspectives on recruitment and retention. Key motivations to initially attend diabetes SMAs were their convenience and affordability, having a trusted referring provider, being encouraged to attend by a loved one, and having an unmet need for effective diabetes education. These findings align with other research, including studies on the benefits of provider referrals (39,40) and partner encouragement (41,42) to attend and the drive to feel more empowered in diabetes self-management (37). Reasons to stay engaged in diabetes SMAs included receiving social support and individual attention while also receiving valuable information for diabetes self-management. Although we previously reported that patients also desired care from a multidisciplinary team and peer mentors to support their attendance in diabetes SMAs (29), qualitative findings in this study did not indicate that these features influenced attendance. This finding is consistent with the comparable attendance that was observed between the patient-driven and standardized SMAs.

There was also no indication from the qualitative interviews that cohort size influenced attendance (i.e., that SMA patients participated more or less because of how many other people were in their cohort). Although quantitative data indicated greater attendance in the smallest (1–9 participants) versus the midsized (10–19 participants) cohorts of diabetes SMAs in this study, attendance rates were comparable for the larger cohorts of ≥20 participants (i.e., still 4 sessions attended on average). In contrast, a prior study of DSMES for older adults found that larger class sizes with ≥20 participants were associated with the lowest attendance levels (43). It is possible that older adults need more individual attention to address medical complexity (44), requiring a smaller class size. Otherwise, larger cohort sizes may be helpful to increase the overall proportion of patients with diabetes who receive services and to optimize the use of limited practice resources, but with consideration of the needs of the specific patient population. Larger cohort sizes have been recommended to optimize revenue and ensure financial sustainability (45).

Qualitative findings suggested that virtual delivery was more engaging for patients with emotional distress or who appreciated the safety and convenience of attending diabetes SMAs from their homes. Preference for virtual SMAs may be especially notable for racial and ethnic minority and low-income populations in FQHCs, who may have especially high levels of diabetes distress (46), heightened concerns about COVID-19 (47,48), and other barriers to traditional in-person care such as lack of transportation and childcare (49,50). In contrast, other patients would have preferred a more enriching experience in diabetes SMAs that could be afforded by in-person delivery, including a greater sense of accountability to the facilitator and other patients, fewer distractions, and the elimination of potential challenges with technology.

Based on these findings, suggestions to improve the reach of diabetes SMAs include 1) expand delivery in FQHCs and similar settings to reach priority populations, 2) offer both in-person and virtual modalities to meet personal preferences, and 3) consider larger cohort sizes to support access and sustainability. Suggestions to increase attendance in diabetes SMAs include 1) provide nutrition education early on, 2) streamline delivery by a facilitator and medical provider to allow for a less resource-intensive but equally engaging intervention, and 3) minimize patient burden and consider additional strategies to retain underserved populations such as providing fee waivers, healthy groceries, and transportation assistance (for in-person programs).

Limitations of this evaluation include a lack of data from patients who were invited to attend diabetes SMAs but declined to enroll or attend, another important dimension of reach (8). Patients who did not respond to offers to participate or did not participate in most of the sessions did not agree to interviews. Hence, views on patient experiences were likely biased toward those who enjoyed SMAs. Interviews revealed little information about why participants did not attend even more in-person sessions, although reasons to either attend or skip virtual sessions were clearer. We did not have access to patient demographics related to income or education, which may have been useful to include in this analysis. However, receiving care in an FQHC versus a non-FQHC practice may serve as a proxy indicator of socioeconomic status. Demographic characteristics (race, ethnicity, sex, and language) also varied between the patient-driven and standardized conditions, which may have been the result of cluster-level rather than individual-level randomization. However, adjusted models controlled for these demographic differences.

In conclusion, diabetes SMAs can reach diverse and underserved populations with adequate attendance. Enhancing delivery with a multidisciplinary care team and a peer mentor and allowing patients to select topic order and emphasis yielded attendance comparable to that of a more standardized approach, suggesting that practices can conserve resources and still deliver engaging services. Delivering diabetes SMAs in FQHCs is an excellent setting to engage priority populations, although further efforts are needed to ensure equitable retention. Increasing access to virtual delivery may be key to ensuring broader reach and facilitating attendance for patients who cannot easily attend or complete in-person programs.

The authors thank the entire Invested in Diabetes project team, including their patient stakeholders and participating practices.

Research in this study was funded through a Patient- Centered Outcomes Research Institute (PCORI) award (IHS-1609-36322). N.D.R. received support from the National Institutes of Health (R01DK119478). The views, statements, and opinions presented in this work are solely the responsibility of the authors and do not represent the official views of these organizations.

No potential conflicts of interest relevant to this article were reported.

N.D.R. contributed to the study conceptualization and investigation, formal analysis, and writing, reviewing, and editing of the manuscript. D.G. contributed to the study conceptualization and investigation, project administration, formal analysis, and writing, reviewing, and editing of the manuscript. M.S. contributed to the study conceptualization and investigation and reviewing and editing of the manuscript. M.C. contributed to the formal analysis and writing of the manuscript. R.E.G. and J.S.H. contributed to the study methodology and investigation and reviewing and editing of the manuscript. J.A.W. contributed to the study conceptualization and investigation, project administration, funding acquisition, and reviewing and editing of the manuscript. B.M.K. contributed to the study conceptualization and investigation, project administration, formal analysis, funding acquisition, supervision, and writing, reviewing, and editing of the manuscript. N.D.R. is the guarantor of this work and, as such, had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

These findings were presented in abstract form at the 15th Annual Conference on the Science of Dissemination and Implementation in Health, 11–14 December 2022, in Washington, D.C.