A Team-Based Online Game Improves Blood Glucose Control in Veterans With Type 2 Diabetes: A Randomized Controlled Trial

Diabetes self-management education (DSME) is a core component of comprehensive quality care of type 2 diabetes (1,2). There is consensus on the appropriate DSME curricular elements (3) but not necessarily how it should be delivered. DSME can generate short-term (<6 months) positive effects on glucose control, but these early gains often dissipate quickly (4–8). A meta-analysis showed that two-thirds of hemoglobin A_1c (HbA_1c) change attributable to DSME is lost in the 1–3 months after completion (9).

A novel form of online education (termed “spaced education” [SE]) based on two robust psychological phenomena (the spacing and testing effects) can improve long-term retention of learning and generate meaningful behavior change (10–12). Delivered via e-mail or mobile application (app), SE presents clinical case scenarios accompanied by multiple-choice questions. Participants are asked to submit answers and are then immediately presented with the correct answer(s) and an explanation of the topic. The material is represented in a cycled pattern to reinforce the content over weeks and months. We have shown in randomized trials involving health care providers that the SE methodology improves knowledge acquisition, boosts learning retention for up to 2 years, and durably improves clinical performance (13–18).

We recently incorporated several game mechanics into SE, including competition among participants and adaptive content delivery that is modified based on a participant’s past performance. Team-based competition within SE games generates significantly stronger engagement among participants (19), and an SE game delivered to clinicians led to significantly improved health measures of their patients (20).

In this study, we hypothesized that a team-based SE game on DSME topics delivered to patients with type 2 diabetes could generate sustained improvements in their HbA_1c. To test this, we conducted a randomized trial with an active control group among veterans with diabetes living in the eastern U.S.

This study was a randomized controlled trial of an online SE game among patients with diabetes. The study received institutional review board approval and was registered at ClinicalTrials.gov (NCT02082704).

The study was conducted from July 2014 to June 2015 among Veterans Affairs (VA) patients in the eastern U.S. We targeted for participation patients with diabetes who had inadequate glucose control while taking oral diabetes medications, with or without the concomitant use of insulin, to study if the DSME game would lower HbA_1c via improved medication adherence. Pharmacy and laboratory data were reviewed for patients with 1) an active prescription for oral diabetes medications as of December 2013 and 2) whose most recent HbA_1c in 2013 was >64 mmol/mol as measured by an in-hospital laboratory. These veterans were solicited via letter and telephone. They asked to enroll by responding affirmatively over the phone or returned a card via mail. Those patients who subsequently provided an e-mail address, completed an online survey, submitted an HbA_1c home test kit (Home Access Inc., Hoffman Estates, IL), and had a home-tested HbA_1c value of ≥58 mmol/mol were included in the study.

The educational content for the SE game was structured in single or multiple correct answer question-explanation format to take advantage of the testing effect. Fifty questions-explanations were developed on the topics of DSME and civics. The DSME content focused on glucose management, exercise, long-term diabetes complications, medication adherence, and nutrition. The civics content was derived from the U.S. Citizenship and Immigration Services Practice Test (https://my.uscis.gov/prep/test/civics). Questions were independently content validated by two diabetes specialists, one primary care physician, and one physician-educator. They were then pilot tested among 30 patients with diabetes, and specific questions were selected for inclusion based on item difficulty, point-biserial correlation, and Kuder-Richardson 20 score. Detailed explanations to accompany each question were constructed to explain the answers, provide a take-home message, and provide references for the content material.

The game used an automated system (Qstream Inc., Burlington, MA) that sent two questions every Tuesday and Thursday to participants via e-mail or mobile app from July to December 2014. Upon answering the multiple-choice question, the patient was immediately presented with the correct answer and a detailed explanation of the question content (Fig. 1). After a question was first presented (round 1), it was sent again 4 weeks later to reinforce learning points (round 2) via the spacing effect. If the question was answered incorrectly in round 1 but correctly in round 2, patients were given the opportunity to receive the question again 2 weeks later in a bonus round. Points were earned based on performance on the questions. Patients were assigned to one of seven teams based on their geographic region. To foster a sense of competition and community, individual and team scores were posted on leaderboards. To protect patients’ identities, each was assigned an alias. Team scores were calculated as the average score of all of its members. In addition, patients were shown how others had answered each question and how many had answered it correctly. Although rewards were structured to take advantage of the demonstrated efficacy of group-based financial incentives (21), we also included individual-based awards to maintain engagement among participants on lower-performing teams. At the end of the game, all members of the two teams with the most points in each arm received a $100 gift certificate to an online store. In addition, the top 30% of individual patients on the remaining teams in each arm received a $100 gift certificate.

Patients were stratified by home-tested HbA_1c level (≥69 vs. <69 mmol/mol) and pill possession ratio (PPR) (≥0.60 or <0.60) and block randomized at a single time point into two arms (block size = 4) using a random allocation sequence (22). All patients began the study simultaneously in July 2014. We provided the identical curricula (DSME and civics) to all participants with only the method of delivery differing between arms (Fig. 2). Patients in the DSME arm received the SE game on DSME and a paper booklet containing the question-explanation materials on civics. Patients in the civics arm received the SE game on civics and a paper booklet containing the question-explanation materials on DSME.

At enrollment and at 6 and 12 months after the launch of the game, patients were sent a home test kit and asked to return a fingerstick blood sample blotted on filter paper and urine specimen by mail. These specimens were assayed by a commercial laboratory for HbA_1c (coefficient of variation: 3.0–3.9%), urine albumin, and urine creatinine (Home Access Inc.). Patient information was withheld from the outside laboratory through use of coded identifiers. In addition, patients completed two online questionnaires: an 8-item Diabetes Empowerment Scale (DES) Short Form (23) and 20-item Problem Areas in Diabetes (PAID) questionnaire (24), at baseline, 6 months, and 12 months. Patients were sent a $75 gift certificate at each of these time points. Additional clinical data were extracted from the VA Corporate Data Warehouse.

Our primary outcome measure was the level of HbA_1c over time. Secondary outcome measures included PPR of oral diabetes medications, urine microalbumin-to-creatinine ratio, and DES and PAID scores.

Enrolling 450 participants provided 0.8 power to detect a 4 mmol/mol difference in HbA_1c between arms at a two-sided 0.05 significance level, assuming an SD of 14 mmol/mol.

PPR was generated for oral diabetes medications during the intervention and follow-up periods and was calculated by taking the total number of day’s supply that the patient received over the course of the study, including any fills that overlapped the beginning or end of the study period, divided by the time between the first fill date and the fill date of the first prescription outside of the study period. If the patient was on multiple diabetes drugs, PPRs were averaged across drugs taken in the same study period. Charlson comorbidity indices, Elixhauser comorbidity indices, and Diabetes Complications Severity Scale scores were calculated using ICD-9 coding algorithms (25–27).

Generalized estimating equations with unstructured correlation matrices were used to examine the association between arm and time with each of the outcomes. The urine microalbumin-to-creatinine ratio was log transformed for the analysis. Intent-to-treat analyses were conducted for the entire study population. For each arm, the generalized estimating equation models estimated mean outcome values as well as mean change in values. Pairwise comparisons were made between arms using the contrast/estimate statements in PROC GENMOD (SAS Institute Inc., Cary, NC). In these analyses, we assumed an ignorable missing data situation (“missing completely at random”) (28,29) and performed the analysis with an unbalanced design using all available information.

As there was missing outcome data at the 6- and 12-month assessment times, we used mixed linear regression models to explore the associations under a “missing at random” assumption. We also used these models to assess the role of “team” as a random effect, to compare models with fixed sample return times (baseline, 6 months, and 12 months) versus actual sample return times, and to compare models with various spatial correlations versus a simple unstructured correlation. Given that the adjusted mixed-model HbA_1c outcomes were similar to those from the initial unadjusted generalized estimating equation model, we present the results of our outcome measures from the initial unadjusted model. In addition to the prespecified analyses, we also present several post hoc analyses to explore possible explanations for observed associations. Post hoc linear regression models were used to obtain correlations between change in DES score, PAID score, and points achieved during the game as predictors of change in HbA_1c values by study arm. As these were not preplanned analyses, we emphasize the exploratory nature of these results. Data were managed using Microsoft SQL Server Management Studio 2014 and Microsoft Excel 2010. All analyses were conducted with SAS 9.2 (SAS Institute Inc.) at the α = 0.05 significance level.

Of 15,455 potentially eligible patients who were contacted, 2,300 (15%) requested to enroll. Among these, 456 (20%) completed the enrollment process, met the inclusion criteria, and were randomized to DSME and civics game arms (227 and 229 patients, respectively) (Supplementary Fig. 1). Demographic characteristics of randomized patients were similar between arms (Table 1). The game was started by 227 (100%) and 225 (98%) of DSME and civics game patients, respectively, with 213 (94%) and 207 (90%) submitting responses to all 50 questions.

Over the 12 months of the study, DSME game patients had significantly greater reductions in mean HbA_1c than civics game patients (−8 mmol/mol [95% CI −10 to −6] and −5 mmol/mol [95% CI −7 to −3], respectively; P = 0.048), with the difference between cohorts manifesting primarily in the 6 months after the games (Fig. 3 and Table 2). Results were similar when adjusting for team and missing data (−8 mmol/mol [95% CI −10 to −6] and −5 mmol/mol [95% CI −7 to −3], respectively; P = 0.051) (Table 2).

Baseline HbA_1c values were higher among those with lower DSME knowledge, as measured by their initial answers to the DSME game questions (β = −0.19, P = 0.005). In contrast, reductions in HbA_1c over the trial were not significantly associated with improvement in DSME knowledge over the course of the game (as measured by final score adjusting for baseline score), with patients’ engagement in the game (as measured by the number of questions answered), and their overall performance on the game (as measured by game points).

Empowerment (as measured by DES) increased among the DSME game patients in the 6-month intervention period (+1.7 [95% CI 0.7 to 2.7]), whereas that of the civics patients decreased (−0.1 [95% CI −1.1 to 0.8], respectively; P = 0.010). There were also greater reductions in diabetes distress (as measured by PAID) among DSME game patients during the intervention period, but this difference was not statistically significant. During the 6-month follow-up period and the 12-month study overall, changes in empowerment and distress scores did not differ significantly between arms.

Changes in PPR and urine microalbumin-to-creatinine ratio during the intervention and follow-up periods showed no significant differences by arm. Similarly, there was no significant difference between arms in the number of diabetes medications started or discontinued during the intervention and follow-up periods.

In patients with baseline HbA_1c levels >75 mmol/mol, HbA_1c reductions over the 12 months were significantly greater among DSME game patients than among civics game patients (−16 mmol/mol [95% CI −21 to −12] and −9 mmol/mol [95% CI −14 to −5], respectively; P = 0.031) (Fig. 3). Among DSME game patients, the number of points earned during the game (mean 14,118 [SD 2,122]) correlated significantly with increases in empowerment (mean 1.12 [SD 7.79], R² = 0.029, P = 0.017), but there was no significant correlation between this increase in empowerment and improvements in HbA_1c.

Our results show that an SE game delivering DSME content to patients with diabetes over a broad geographic area generates short-term increases in patient empowerment and longer-term improvements in HbA_1c, compared with a control intervention (civics game and paper-based DSME). Among DSME game patients with elevated HbA_1c at baseline (>75 mmol/mol), the overall reduction in HbA_1c is comparable to that of starting a new diabetes medication (30,31). In aggregate, our study provides rigorous evidence that an online game administered among patients with diabetes can generate meaningful and sustained improvements in their glucose control.

It is noteworthy that the greatest impact of the DSME game on HbA_1c was seen in the 6 months after completing the game. This delayed impact may be due to the time lag between the gradual adoption of appropriate health-improving behaviors induced by the DSME game and the glycemic improvements eventually reflected in HbA_1c. By its very nature, the spacing effect requires learning to be distributed over spaced intervals of time. Educational programs structured to take advantage of the spacing effect have been shown to generate knowledge and skills that are retained more effectively over time, but this SE learning process is gradual (13,15).

Our initial hypothesis was that HbA_1c would be impacted via increased adherence to oral diabetes medications, but this was not supported by our findings. The DSME game may have affected longer-term glucose control via lifestyle changes such as exercise and nutrition, but we did not capture these data from patients. In future research, mobile apps to track lifestyle changes used in combination with the DSME game may help define the role of exercise and nutrition in the longer-term HbA_1c changes we observed (32). The financial incentives were similar in both arms and thus were unlikely to have caused the differential benefit on HbA_1c. In an attempt to elucidate other potential mechanisms of action, we performed several exploratory analyses. The number of points earned in the DSME game, which reflects patients’ engagement with the game as well as their baseline and acquired DSME knowledge, had a small but significant correlation with changes in empowerment. We also noted a significant increase in empowerment among DSME game patients during the intervention period that preceded their significant improvement in HbA_1c but could not demonstrate a significant relationship between this increase in empowerment and improvements in HbA_1c.

Of note, the control group also sustained substantial reductions in HbA_1c. This is not unexpected based on our prior research showing that DSME booklets can significantly improve HbA_1c over 12 months (33). These HbA_1c changes may also reflect an observation effect caused by game participation, regression to the mean, and/or patients’ engagement in virtual communities generated through the team-based competition. These findings highlight the importance of including an adequately designed control group to clarify the true impact of patient-directed interventions.

As technology has advanced over the last 2 decades, multiple attempts have been made to harness interactive computer games to improve self-management of patients with diabetes, shifting from CD-ROMs to game consoles (e.g., PlayStation; Sony Corporation, Tokyo, Japan) to the Internet (34). Overall, data supporting the efficacy of these interventions are extremely limited (35). Most studies focus on collecting information on user experience, frequently lack a control group, and fail to assess longer-term outcomes. Among randomized trials of computer-based DSME interventions that do assess longer-term outcomes, a recent meta-analysis found that there was no significant difference in HbA_1c between study arms when patients were tested 6 months or more after the interventions (36). Similarly, a meta-analysis of game-based interventions for patients with diabetes found only three studies that were of sufficient methodologic rigor for inclusion, and these games showed no effect on HbA_1c compared with control subjects (37).

Our study has a number of strengths, including the novelty of the game-based intervention, a focus on longer-term health outcomes, geographic diversity of the patient population, and a study design that doubly controlled for game participation and content exposure. In addition, our study was conducted remotely without requiring physical contact with patients and used in-home testing and administrative data for outcome assessments, providing evidence for scalability of such interventions to larger patient populations and health care systems. One benefit of the SE methodology is that participants merely require a computer or mobile device with an Internet connection. Such an intervention could be deployed in geographically dispersed and remote health care settings (38). We are developing an implementation strategy to deploy the DSME game initially in the VA New England Healthcare System and then to expand across the Veterans Health Administration.

Our findings must also be considered in the context of several limitations. Enrollment in the trial required Internet and e-mail access, thereby enriching the study with patients who are comfortable using technology. This may limit its generalizability, as may the predominantly male veteran population from which we recruited. Although a small percentage of the eligible patients who were contacted wanted to enroll and fewer were randomized, these participation rates are consistent with both our prior studies and with real-world experience with recruiting participants into weight loss and diabetes prevention programs (39–41). We were not able to reliably assess the degree to which patients engaged with the paper-based DSME materials and thus cannot determine if and how this impacted HbA_1c levels among the civics game participants. The between-group difference in HbA_1c is significant but small, albeit this improvement is considered at the threshold of clinical significance by the U.S. Food and Drug Administration (42). In addition, we collected limited outcome data from study participants, and these did not provide evidence of a clear mechanism to explain the effects of the intervention. Finally, we do not have health outcomes data beyond 12 months, so we cannot determine the degree to which the improvements in HbA_1c are durable. Further research is needed to determine if and how such games can maintain or increase improvements in HbA_1c over time.

In summary, our randomized trial demonstrates that an SE game delivering DSME content generates significant improvements in HbA_1c over 12 months among patients with diabetes. This methodology may be an effective and scalable method by which to improve health outcomes in patients with diabetes and other chronic diseases.

Acknowledgments. The authors recognize the invaluable work of Rebecca Lamkin, Antoun Houranieh, and Nicole Kosik of the Veterans Affairs Boston Healthcare System.

Funding. This study was supported by the Agency for Healthcare Research and Quality (1R01HS019708-01A). The study protocol received approval from the Veterans Affairs Boston Healthcare System institutional review board. This material is the result of work supported with resources and the use of facilities at the Veterans Affairs Boston Healthcare System.

The funding sources had no role in the design and conduct of the study, collection, management, analysis, and interpretation of the data, preparation, review, or approval of the manuscript, and decision to submit the manuscript for publication.

Duality of Interest. B.P.K. is an equity owner of Qstream Inc., an online platform launched by Harvard University to host SE outside of its firewalls. No other potential conflicts of interest relevant to this article were reported.

Author Contributions. B.P.K. designed the study, acquired data, and drafted the manuscript. D.R.G. designed the study, acquired data, performed statistical analyses, and drafted the manuscript. G.T.M. and J.D.O. designed the study and critically revised the manuscript for important intellectual content. K.E.K. acquired data, performed statistical analyses, and drafted the manuscript. P.R.C. designed the study and drafted the manuscript. All authors interpreted data. D.R.G. is the guarantor of this work and, as such, had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.