OBJECTIVE

Internet-based secure messaging between patients and providers through a patient portal is now common in the practice of modern medicine. There is limited evidence on how messaging is associated with use and clinical quality measures among patients with type 2 diabetes. We examine whether messaging with physicians for medical advice is associated with fewer face-to-face visits and better diabetes management.

RESEARCH DESIGN AND METHODS

Patients with diabetes who were enrolled in an online portal of an outpatient health care organization in 2011–2014 were studied (N = 37,762 patient-years). Messages from/to primary care physicians or diabetes-related specialists for medical advice were considered. We estimated the association of messaging with diabetes quality measures, adjusting for patient and provider characteristics and patient-level clustering.

RESULTS

Most patients (72%) used messaging, and those who made frequent visits were also more likely to message. Given visit frequency, no (vs. any) messaging was negatively associated with the likelihood of meeting an HbA1c target of <8% (64 mmol/mol) (odds ratio [OR] 0.83 [95% CI 0.77, 0.90]). Among message users, additional messages (vs. 1) were associated with better outcome (two more messages: OR 1.17 [95% CI 1.06, 1.28]; three more messages: 1.38 [1.25, 1.53]; four more messages: 1.55 [1.43, 1.69]). The relationship was stronger for noninsulin users. Message frequency was also positively associated, but to a smaller extent, with process measures (e.g., eye examination). Physician-initiated messages had effects similar to those for patient-initiated messages.

CONCLUSIONS

Patients with diabetes frequently used secure messaging for medical advice in addition to routine visits to care providers. Messaging was positively associated with better diabetes management in a large community outpatient practice.

With the advance in communication technology using the internet and mobile devices in recent years, internet-based secure messaging between patients and providers through a “patient portal” is becoming a common practice in modern medicine (1,2). Although this type of internet-based communication was launched mainly to meet patient demands for convenience (3), there may be additional benefits in chronic disease management. Increased frequency of contact using telephone-based interventions, in addition to face-to-face visits, have shown improved outcomes for patients with diabetes (4). Remote communication with providers using secure online messaging does not require patients’ time, effort, or cost to travel to clinicians’ offices, and the asynchronous nature of such communication can improve access and convenience by allowing patients and clinicians to initiate communication at any time (5,6).

Leaders in medicine have recommended (710) that the locus of primary care and chronic care delivery should expand beyond traditional office visits to include alternative modes of communication. Over-reliance on technology that leaves little room for in-person physical examination and consultation, the most powerful tools for diagnosis, can compromise in-person interactions and be harmful to the patient-provider relationship (11,12). However, for managing chronic conditions, effective use of technology to assist and complement limited time for in-person visits may augment the personal aspect of patient-physician interaction. Given the limited face-to-face time with providers during office visit time and ongoing needs for care management, between-visit electronic contacts may help manage diabetes.

There is limited evidence on how secure messaging affects patient-physician face-to-face encounters and clinical quality of care among patients with type 2 diabetes. Messaging with providers could be used as a substitute for traditional office visits and thus be cost saving for patients and payers (5). Studies of patients with diabetes found that increased use of outpatient and inpatient care (1315) improved diabetes care processes and outcomes (1620) associated with the use of patient portal or secure messaging with providers. However, these studies were conducted mainly as clinical trials of messaging linked with diabetes management programs or in integrated delivery organizations, which may not be representative of how messaging is used in community settings. They also did not explore these relationships in subgroups of patients with additional communication needs (e.g., language barriers) who may be more likely to benefit from messaging.

We examine how patients’ use of secure electronic messaging is associated with the management of type 2 diabetes. Our main research questions are whether and for whom the use of secure messaging with physicians for medical advice is associated with fewer face-to-face visits and improvements in the clinical outcomes of diabetes. In addition to the overall association, we assessed the relationship of secure messaging across patients with varying needs in terms of frequency of visits, age, number of chronic conditions, and English preference.

Study Setting

We used the natural variation in message usage among patients as well as across physicians. We used data from the electronic health records of a large, mixed-payer, multispecialty outpatient care group practice organization in northern California. The organization implemented a secure messaging system embedded in the online patient portal in 2002 (21). It became freely available to anyone in 2011. However, not every patient used these services because enrollment in an online portal account is voluntary and, of those who set up an account, many did not actively use the portal or secure messaging.

Study Cohort

Primary care patients with diabetes between 18 and 75 years of age in 2011–2014 who were enrolled in the patient online portal were studied. Primary care patients were defined each year as those who made at least one primary care visit during the year. Patients with diabetes were identified as those who carried a diagnosis of diabetes in their electronic health record problem list (or current problem).

Measures

Types of Messages Studied

Messages sent through a secure online patient portal with a primary care physician (PCP) or diabetes-related specialist (endocrinology, nephrology, ophthalmology, podiatry, or nutrition) were included. In terms of topic area, we selected for “medical advice request” messages only, which were reviewed and answered by clinicians, usually a physician. Nonclinical communication related to other topics or messages often handled by medical assistants or midlevel clinicians (e.g., appointment request or reminder, routine medication renewal requests, or queries for scheduling, billing-related, or technical issues) were excluded.

Message and Visit Frequency

Multiple messages under a “subject” were bundled as one “thread” and considered as a unit when counting the frequency of messages. When a subsequent message under the same subject followed in >4 days, the message and subsequent messages were treated as another message. On average, there were 2.2 message exchanges within a thread. We also identified whether the initial message was sent by a patient versus a physician. Visit frequency was defined as the number of visits to a PCP or diabetes-related specialist, as listed above.

For visit frequency, indicators of two, three, and four or more visits per year, respectively, were included in the regression analysis. Note that we studied active primary care patients who made a visit to a PCP or diabetes-related specialist, and thus the referent category for office visit frequency was 1 rather than 0. We then created a comparable set of variables for messaging with indicators of two, three, and four or more messages per year, with the referent category of one message per year, in addition to a variable indicating no message (vs. any message).

Clinical Outcomes

We assessed several indicators of diabetes management in the Healthcare Effectiveness Data and Information Set (HEDIS) established by the National Committee for Quality Assurance (NCQA) and endorsed by the American Diabetes Association (22,23). This included HbA1c screening and control (<8% [64 mmol/mol]), blood pressure control (<130/90 mmHg), eye examination, and medical attention for nephropathy.

Analytical Approach

We first used bivariate analysis to compare demographic and clinical characteristics between users and nonusers of secure messaging, using a t test and χ2 test. We used multivariate logistic regressions at the patient-year level. As one patient could be included up to four times in the 4-year period of analysis, we accounted for patient-level clustering using Huber-White SEs.

The following covariates were taken into account: frequency of visits and messages, patient age, sex, race/ethnicity, English proficiency, insulin use, comorbidities, insurance type, number of years with own PCP, as well as PCP sex, years of medical practice, department/specialty, and number of patients with diabetes. In addition to the overall model, we conducted a subgroup analysis based on age (18–64 vs. ≥65 years), insulin use (any vs. no insulin prescription during a year), number of chronic conditions (one to two vs. three or more), and preferred language (English vs. other). Based on the parameters from the regression model, we estimated the association between incremental change in messaging and office visit and clinical outcome measures.

One methodological concern is that people who use secure messaging may be more motivated patients and thus are more likely to have better clinical outcomes solely through better self-management. Such a selection issue may be reduced, but not eliminated, by limiting the study to those who were signed up for the online portal. To further address the issue, in sensitivity analysis, we assessed the possibility of selection in two ways. First, we distinguished messages initiated by a physician from those initiated by a patient. A significant and comparable effect of physician-initiated (vs. patient-initiated) messaging suggests an independent association due to the online communication itself, rather than being due solely to patient selection. Second, we estimated an alternative model using an individual fixed-effects model. In an individual fixed model, only “within-individual” variations over time in visit or message frequency and clinical outcomes are used to identify the relationship between the two, because each person (at a different time point) serves as his/her own control. We also ran a difference-in-differences model where the outcomes of the messaging patients, relative to the nonusers, was compared before and after 1 January 2012 (the period after 2012–2014 included a large expansion of secure messaging). This model provides another approach to mitigating the effects of selection bias by comparing differences between the two groups over time. Statistical significance was determined at P < 0.05. All of the analysis was performed in Stata MP version 13.1 (College Station, TX). The study was approved by the Institutional Review Board at the health care organization.

Demographic and Clinical Characteristics of Message Users and Nonusers

A total of 20,655 unique patients (with diabetes and enrolled in a patient online portal) who made a visit to 394 unique providers in 2011–2014 were included in the study. Of those in 2014 (N = 15,316), the average age was 57.2 years, 44% were female, 21% received a prescription for insulin, and the average Charlson Comorbidity Index (24) (not counting age) was 1.9 (range 1–14) (Table 1). Patients made 3.8 visits on average to a provider during the year. See Table 1 for more details in patient and provider characteristics. Overall patient and provider characteristics and visit frequencies were similar in other years (Supplementary Table 1) (25,26).

Table 1

Patient and provider characteristics (2014)

VariablesOverall (N = 15,316)Message user (N = 12,485)Message nonuser (N = 2,831)
Visit and messaging frequency (minimum-maximum)  
 Any message 82   
  Message frequency among users (1–56)  5.3 (5.5)  
 Any patient-initiated message 65   
  Patient-initiated message frequency, if any (n = 9,625) (1–40)  3.3 (4.0)  
 Any provider-initiated message 63   
  Provider-initiated message frequency, if any (n = 9,691) (1–31)  2.0 (2.1)  
 Visits to a PCP or diabetes-related specialist (1–45) 3.8 (3.0) 3.9* (3.1) 3.2 (2.6) 
Patient characteristics (minimum-maximum)  
 Age, years (18–75) 56.4 (11.7) 56.3 (11.7) 59.1 (11.7) 
 Female 43 44* 40 
 Insulin prescription 21 22* 16 
 Comorbidities (Charlson comorbidity index without age) (1–14) 1.9 (1.3) 1.9 (1.3) 1.8 (1.3) 
 Race/ethnicity    
  Non-Hispanic white 40 41* 35 
  Black 
  Asian 32 32 33 
  Latino 4* 
  Other race or race not reported 21 20 21 
 Non-English language preferred 
 Insurance type    
  Preferred provider organization plan 78 78* 79 
  Health maintenance organization plan 19 20* 18 
  Other insurance 
 Years with the PCP (0–15) 4.9 (3.7) 4.9* (3.8) 4.7 (3.3) 
Provider characteristicsa  
 Female 58 58* 54 
 Number of years of practice (4–45) 20.6 (8.9) 20.4* (8.8) 21.3 (9.3) 
 Number of patients with diabetes (1–178) 91.4 (44.3) 91.1 (44.8) 92.8 (42.2) 
VariablesOverall (N = 15,316)Message user (N = 12,485)Message nonuser (N = 2,831)
Visit and messaging frequency (minimum-maximum)  
 Any message 82   
  Message frequency among users (1–56)  5.3 (5.5)  
 Any patient-initiated message 65   
  Patient-initiated message frequency, if any (n = 9,625) (1–40)  3.3 (4.0)  
 Any provider-initiated message 63   
  Provider-initiated message frequency, if any (n = 9,691) (1–31)  2.0 (2.1)  
 Visits to a PCP or diabetes-related specialist (1–45) 3.8 (3.0) 3.9* (3.1) 3.2 (2.6) 
Patient characteristics (minimum-maximum)  
 Age, years (18–75) 56.4 (11.7) 56.3 (11.7) 59.1 (11.7) 
 Female 43 44* 40 
 Insulin prescription 21 22* 16 
 Comorbidities (Charlson comorbidity index without age) (1–14) 1.9 (1.3) 1.9 (1.3) 1.8 (1.3) 
 Race/ethnicity    
  Non-Hispanic white 40 41* 35 
  Black 
  Asian 32 32 33 
  Latino 4* 
  Other race or race not reported 21 20 21 
 Non-English language preferred 
 Insurance type    
  Preferred provider organization plan 78 78* 79 
  Health maintenance organization plan 19 20* 18 
  Other insurance 
 Years with the PCP (0–15) 4.9 (3.7) 4.9* (3.8) 4.7 (3.3) 
Provider characteristicsa  
 Female 58 58* 54 
 Number of years of practice (4–45) 20.6 (8.9) 20.4* (8.8) 21.3 (9.3) 
 Number of patients with diabetes (1–178) 91.4 (44.3) 91.1 (44.8) 92.8 (42.2) 

Values are mean (SD) or percentages. The statistics in this table are from patients in 2014 only. Message frequencies are much higher in 2014 than from 2011 to 2013, respectively. Frequencies for other patient and provider characteristics are similar to other years.

*P < 0.01.

aPCP or diabetes-related specialist that the patient saw the most during the year.

Compared with those patients who were enrolled in the online portal but did not send any messages during a year, messaging users were more likely to be younger, female, and non-Hispanic white and to have had a longer relationship with their own PCP. On average, message users made 3.9 visits per year to a PCP or diabetes-related specialists, whereas message nonusers made 3.2 visits during the year. These differences were statistically significant (P < 0.01).

Message Frequency and Its Relationship With Visit Frequency

Over the 4 years, there was a steep increase in the proportion of online portal enrollees who exchanged messages with a provider, from 45% in 2011 and 73% in 2012 to 82% in both 2013 and 2014. Overall, 72% of patients (N = 37,762 patient-years) sent some messages during a year, 15% sent one message, 12% sent two messages, 9% sent three messages, 7% sent four messages, and 28% sent five or more messages. Messaging users exchanged on average 4.9 message threads (i.e., messages) per year (SD 4.7; range 1–59 messages), and a majority were patient-initiated messages (4.0 per year) compared with provider-initiated messages (2.4 per year). The frequency of messaging among users also increased from 3.9 per year (2011) to 5.3 per year (2014).

Heavy message users (five or more messages per year) also made more frequent visits (4.8 visits a year) than no-message users (3.2 per year) (P < 0.01) (Fig. 1). Moderate message users (two to four messages per year) were also likely to make slightly more visits (3.3–3.6 visits) than no-message users (P < 0.01).

Figure 1

Relationship between messaging and visit frequencies. Differences among messaging categories: 1, 2, 3, 4, and ≥5 messages (vs. 0) are statistically significant at P < 0.01.

Figure 1

Relationship between messaging and visit frequencies. Differences among messaging categories: 1, 2, 3, 4, and ≥5 messages (vs. 0) are statistically significant at P < 0.01.

Relationship Between Messaging and Visit Frequency and Clinical Outcomes

In multivariate regression, increased frequency of messaging was positively associated with HbA1c control. The odds ratios (ORs) were 1.17 (95% CI 1.06, 1.28) for two messages per year, 1.38 (1.25, 1.53) for three messages per year, and 1.55 (1.43, 1.69) for four or more messages per year compared with one message per year (all at P < 0.01) (Fig. 2). No message (vs. any message) was negatively associated with the outcome (OR 0.83 [95% CI 0.77, 0.90], P < 0.01). (See Supplementary Fig. 1 for the relationship beyond four messages per year). To a comparable but smaller, degree, increased visit frequency was positively associated with HbA1c control (range: OR 1.16 [95% CI 1.08, 1.25] for two visits per year and 1.29 [1.18, 1.40] for three visits per year to 1.46 [1.35, 1.58] for four visits or more per year, compared with one visit a year). See Supplementary Table 2 for OR (95% CI) values for other predictors.

Figure 2

Effect of visit and messaging frequencies on the likelihood of meeting an HbA1c target of <8% (64 mmol/mol). In the multilevel regression analysis, patient-level factors (age, sex, insulin prescription, number of comorbidities, race/ethnicity, language preference, insurance type, and number of years with primary care provider) and provider-level factors (sex, number of years of practice, number of patients with diabetes) and patient-level clustering are adjusted. ref, reference.

Figure 2

Effect of visit and messaging frequencies on the likelihood of meeting an HbA1c target of <8% (64 mmol/mol). In the multilevel regression analysis, patient-level factors (age, sex, insulin prescription, number of comorbidities, race/ethnicity, language preference, insurance type, and number of years with primary care provider) and provider-level factors (sex, number of years of practice, number of patients with diabetes) and patient-level clustering are adjusted. ref, reference.

The association between visit and message frequency and diabetes care persisted, to a smaller degree, in other quality metrics, which focused more on “timely completion of recommended procedures” including screening of HbA1c, eye examination, and monitoring of nephropathy (Supplementary Table 3).

The association was similar for patient-initiated versus physician-initiated messages (Supplementary Table 4). Further, an increased frequency of messaging had a significant association with HbA1c control in an individual fixed-effects model in which any unobserved individual characteristics (e.g., motivation) was taken into account by using the same individual as its own control (Supplementary Table 5). The difference-in-differences model suggested that users of secure messaging were better at achieving the HbA1c target and that the difference between users and nonusers was larger during the period in which messaging was widely used (Supplementary Table 6).

The interactive relationship between messaging and visit frequency and HbA1c control is plotted in Fig. 3. At any visit frequency, the predicted probability of meeting the guideline recommendation increased with additional messages, as indicated by a nonoverlapping CI. For messaging nonusers, additional visits did not fully replace the additional messages in meeting the HbA1c recommendation. For example, patients who made one visit and used three or more messages (predicted probability ± 95% CI = 83–84%) were significantly more likely to meet the target than those who made four visits and sent no messages (80–82%).

Figure 3

Likelihood of meeting an HbA1c target of <8% (64 mmol/mol) with increasing frequency of messaging, given visit frequency. In the multilevel regression analysis, patient-level factors (age, sex, insulin prescription, number of comorbidities, race/ethnicity, language preference, insurance type, number of years with primary care provider) and provider-level factors (sex, number of years of practice, number of patients with diabetes) and patient-level clustering are adjusted.

Figure 3

Likelihood of meeting an HbA1c target of <8% (64 mmol/mol) with increasing frequency of messaging, given visit frequency. In the multilevel regression analysis, patient-level factors (age, sex, insulin prescription, number of comorbidities, race/ethnicity, language preference, insurance type, number of years with primary care provider) and provider-level factors (sex, number of years of practice, number of patients with diabetes) and patient-level clustering are adjusted.

The relative importance of messaging versus visit frequency differed across subgroups. Increasing the message frequency had a stronger and greater association with HbA1c control than increased visits among younger (age ≤64 years) patients whose preferred language was English and insulin users. The opposite (i.e., visit frequency had a stronger association with HbA1c control than message frequency) was true for those who were ≥65 years of age, had a preferred speaking language other than English, or were insulin nonusers (Supplementary Table 7).

We used data from a real-world outpatient practice setting to estimate the effect of secure messaging with providers of medical advice on measures of diabetes care. We found that secure messaging was associated with better clinical processes and outcomes of diabetes care with a dose-dependent relationship such that an additional message was beneficial for up to three messages per year. Messaging use did not appear to replace face-to-face visits among frequent visitors of the health care system, who were also more likely to have complex health problems, but it was associated with slightly fewer visits among infrequent users of the health care system. Overall, the association between an additional message and outcomes was larger than that of an additional in-person visit among those who saw providers at least annually. Provider-initiated messages had an effect size that was comparable to that of patient-initiated messages, suggesting that patient motivation may not be not the main driver of the positive association.

Our findings were significant in a diverse and heterogeneous population, but outcomes varied across subpopulations. Messaging had a stronger association with HbA1c outcomes for patients whose preferred spoken language was not English. Electronic communication may have complemented a limited duration for in-person office visit time for patients who were ≤64 years of age and were likely to be employed. Finally, Insulin users may have received a greater benefit from messaging because of their disease severity and need for close monitoring.

Our findings were largely consistent with those of previous studies conducted in a variety of practice settings. A well-designed intervention study (18) demonstrated the effectiveness of a web-based intervention for improving diabetes outcomes, but the study assessed the effect of a resource-rich, short-term intervention on targeted, selective participants. Thus, it does not describe the effectiveness of secure messaging in community-based outpatient settings. In a study conducted in an academic setting with a small number of patients, messaging was associated with improved HbA1c control but increasing the frequency of messaging was not associated with better control (17). This study did not account for a nonlinear dose-response relationship, however. Finally, a study conducted in a large integrated outpatient care setting (14) also showed that messaging was associated with better glycemic control and a higher rate of HbA1c testing adherence.

One earlier study (19) using data in the same setting as ours reported that secure messaging was associated with improved processes of care (e.g., HbA1c screening) but was not consistently associated with clinical outcomes (e.g., HbA1c control). There are several differences between that study and ours, which may explain the different conclusion. First, the current study classified messaging frequency to assess a nonlinear dose-response relationship. That is, we found that the incremental effect of messaging was no longer significant when the messaging frequency was more than three messages per year. Second, our study focused on messages answered by a physician or specialist, whereas the previous study included messages answered by the providers, medical assistants, or other care team members. Third, we used data in more recent years among all patients with diabetes at the study setting. In order to tease apart potential selection, the previous study analyzed the effect of patient-initiated messaging among the subset of patients with active portals who did not use the messaging service when it was associated with a nominal fee. Instead, we compared the effect of patient-initiated versus provider-initiated messages to assess the extent to which patient motivation might have driven the effect of messaging on clinical outcomes.

Although the positive association between messaging and clinical outcomes in a community practice setting is encouraging, it is unclear whether free secure messaging is financially sustainable in a predominately fee-for-service environment. The organization in this study used an internal discretionary fund to compensate physicians with $3 to $5 per message thread when the messaging service became free to patients in 2011. This arrangement may not be feasible in the long run, even with the associated increase in office visits or in more resource-constrained settings.

Our findings require some qualification. First, the limitations of a retrospective observational study design prevent us from making casual claims about the impact of messaging. However, we found that patient-initiated and physician-initiated messages had similar and significant effect sizes and that the association was still significant after controlling for unobserved patient characteristics through individual fixed-effects and difference-in-differences approaches. This provides some evidence that our findings may not be driven by patient motivation or patient selection. A future multicenter randomized controlled trial would provide more robust evidence. Second, our study population included patients who were fully insured and saw a PCP or received diabetes-related care at least once a year. However, our subgroup analysis found that messaging might be more effective among patients whose primary language was not English, suggesting that future studies should focus on other underserved populations. Third, the patient education level, which was not included in the study because of a lack of data, may be a confounder between users and nonusers of messaging. However, our population consisted disproportionately of people with a college or higher degree, so any variation due to education level may have been limited. Fourth, our study did not include patient- or provider-reported experiences. Although patients are generally in favor of messaging (2729), providers have expressed mixed opinions (3032). Last, we did not study the content of the messages or visits. Classifying visits and messages based on text information is complex and beyond the scope of this study as it requires natural language processing. Future studies investigating the content of visits and messages may find a stronger association with clinical outcomes if only those diabetes-related visits or messages are studied.

In conclusion, patients with diabetes frequently used secure online messages for medical advice in addition to their routine visits to care providers. This additional mode of communication with providers was positively associated with better diabetes management through improved access to care, particularly for patients with additional communication needs (e.g., language barrier, multiple chronic conditions). Although the effect of messaging on the provider experience and the total cost of care has yet to be determined, secure messaging appears to be used as a tool to complement, rather than replace, in-person office visits and has a stronger beneficial association among patients with additional needs for communication.

See accompanying article, p. 1349.

Acknowledgments. The authors thank Abraham Verghese, Sonoo Thadaney, Donna Zulman (all of Stanford University School of Medicine), and other participants of Stanford Cyber Initiative seminars for valuable comments.

Funding. This study received funding from the Stanford Presence Center in collaboration with the Stanford Cyber Initiative and from Agency for Healthcare Research and Quality (grant K01-HS-019815-01A1 to S.C.).

Duality of Interest. No potential conflicts of interest relevant to this article were reported.

Author Contributions. S.C. conceptualized the study, conducted the analysis, and wrote the manuscript. L.Pan. helped to conceptualize the study and reviewed and edited the manuscript. J.C. reviewed and edited the manuscript. L.Pal. conceptualized the study, interpreted the results, and provided critical review of the manuscript. S.C. is the guarantor of this work and, as such, had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

1.
Kruse
CS
,
Bolton
K
,
Freriks
G
.
The effect of patient portals on quality outcomes and its implications to meaningful use: a systematic review
.
J Med Internet Res
2015
;
17
:
e44
2.
Cronin
RM
,
Davis
SE
,
Shenson
JA
,
Chen
Q
,
Rosenbloom
ST
,
Jackson
GP
.
Growth of secure messaging through a patient portal as a form of outpatient interaction across clinical specialties
.
Appl Clin Inform
2015
;
6
:
288
304
[PubMed]
3.
Williams
ED
,
Bird
D
,
Forbes
AW
, et al
.
Randomised controlled trial of an automated, interactive telephone intervention (TLC Diabetes) to improve type 2 diabetes management: baseline findings and six-month outcomes
.
BMC Public Health
2012
;
12
:
602
[PubMed]
4.
Gregg
EW
,
Chen
H
,
Wagenknecht
LE
, et al.;
Look AHEAD Research Group
.
Association of an intensive lifestyle intervention with remission of type 2 diabetes
.
JAMA
2012
;
308
:
2489
2496
[PubMed]
5.
McGeady
D
,
Kujala
J
,
Ilvonen
K
.
The impact of patient-physician web messaging on healthcare service provision
.
Int J Med Inform
2008
;
77
:
17
23
[PubMed]
6.
Car
J
,
Sheikh
A
.
Email consultations in health care: 1–scope and effectiveness
.
BMJ
2004
;
329
:
435
438
[PubMed]
7.
Margolius
D
,
Bodenheimer
T
.
Transforming primary care: from past practice to the practice of the future
.
Health Aff (Millwood)
2010
;
29
:
779
784
[PubMed]
8.
Committee on the Quality of Health Care in America-Institute of Medicine
.
Crossing the Quality Chasm: A New Health System for the Twenty-First Century
.
Washington, DC
,
National Academy Press
,
2001
9.
Bates
DW
,
Wells
S
.
Personal health records and health care utilization
.
JAMA
2012
;
308
:
2034
2036
[PubMed]
10.
Wagner
EH
,
Coleman
K
,
Reid
RJ
,
Phillips
K
,
Abrams
MK
,
Sugarman
JR
.
The changes involved in patient-centered medical home transformation
.
Prim Care
2012
;
39
:
241
259
[PubMed]
11.
Zulman
DM
,
Shah
NH
,
Verghese
A
.
Evolutionary pressures on the electronic health record: caring for complexity
.
JAMA
2016
;
316
:
923
924
[PubMed]
12.
Chi
J
,
Verghese
A
.
Clinical education and the electronic health record: the flipped patient
.
JAMA
2014
;
312
:
2331
2332
[PubMed]
13.
Liss
DT
,
Reid
RJ
,
Grembowski
D
,
Rutter
CM
,
Ross
TR
,
Fishman
PA
.
Changes in office visit use associated with electronic messaging and telephone encounters among patients with diabetes in the PCMH
.
Ann Fam Med
2014
;
12
:
338
343
[PubMed]
14.
Harris
LT
,
Haneuse
SJ
,
Martin
DP
,
Ralston
JD
.
Diabetes quality of care and outpatient utilization associated with electronic patient-provider messaging: a cross-sectional analysis
.
Diabetes Care
2009
;
32
:
1182
1187
[PubMed]
15.
Palen
TE
,
Ross
C
,
Powers
JD
,
Xu
S
.
Association of online patient access to clinicians and medical records with use of clinical services
.
JAMA
2012
;
308
:
2012
2019
[PubMed]
16.
Shimada
SL
,
Allison
JJ
,
Rosen
AK
,
Feng
H
,
Houston
TK
.
Sustained use of patient portal features and improvements in diabetes physiological measures
.
J Med Internet Res
2016
;
18
:
e179
[PubMed]
17.
Petullo
B
,
Noble
B
,
Dungan
KM
.
Effect of electronic messaging on glucose control and hospital admissions among patients with diabetes
.
Diabetes Technol Ther
2016
;
18
:
555
560
[PubMed]
18.
Ralston
JD
,
Hirsch
IB
,
Hoath
J
,
Mullen
M
,
Cheadle
A
,
Goldberg
HI
.
Web-based collaborative care for type 2 diabetes: a pilot randomized trial
.
Diabetes Care
2009
;
32
:
234
239
[PubMed]
19.
McClellan
SR
,
Panattoni
L
,
Chan
AS
,
Tai-Seale
M
.
Patient-initiated electronic messages and quality of care for patients with diabetes and hypertension in a large fee-for-service medical group: results from a natural experiment
.
Med Care
2016
;
54
:
287
295
[PubMed]
20.
Goldzweig C, Towfigh A, Paige N. Systematic review: secure messaging between providers and patients, and patients’ access to their own medical record. Evidence on health outcomes, satisfaction, efficiency, and attitudes [Internet], 2012. Washington, DC, Department of Veterans Affairs, Health Services Research & Development Service. Available from http://www.hsrd.research.va.gov/publications/esp/myhealthevet-EXEC.pdf. Accessed 17 November 2016
21.
Tang
PC
,
Black
W
,
Buchanan
J
, et al
.
PAMFOnline: integrating EHealth with an electronic medical record system
.
AMIA Annu Symp Proc
2003
;
2003
:
644
648
[PubMed]
22.
National Committee for Quality Assurance (NCQA). HEDIS 2010 Summary Table of Measures [Internet] 2010. Available from http://www.ncqa.org/Portals/0/HEDISQM/HEDIS2014/List_of_HEDIS_2014_Measures.pdf. Accessed 4 August 2017
23.
American Diabetes Association
.
Executive summary: Standards of Medical Care in Diabetes–2010
.
Diabetes Care
2010
;
33
(
Suppl. 1
):
S4
S10
[PubMed]
24.
Charlson
ME
,
Pompei
P
,
Ales
KL
,
MacKenzie
CR
.
A new method of classifying prognostic comorbidity in longitudinal studies: development and validation
.
J Chronic Dis
1987
;
40
:
373
383
[PubMed]
25.
Shanafelt
TD
,
Dyrbye
LN
,
Sinsky
C
, et al
.
Relationship between clerical burden and characteristics of the electronic environment with physician burnout and professional satisfaction
.
Mayo Clin Proc
2016
;
91
:
836
848
[PubMed]
26.
Babbott
S
,
Manwell
LB
,
Brown
R
, et al
.
Electronic medical records and physician stress in primary care: results from the MEMO Study
.
J Am Med Inform Assoc
2014
;
21
:
e100
e106
[PubMed]
27.
Ralston
JD
,
Carrell
D
,
Reid
R
,
Anderson
M
,
Moran
M
,
Hereford
J
.
Patient web services integrated with a shared medical record: patient use and satisfaction
.
J Am Med Inform Assoc
2007
;
14
:
798
806
[PubMed]
28.
Hassol
A
,
Walker
JM
,
Kidder
D
, et al
.
Patient experiences and attitudes about access to a patient electronic health care record and linked web messaging
.
J Am Med Inform Assoc
2004
;
11
:
505
513
[PubMed]
29.
Wallwiener
M
,
Wallwiener
CW
,
Kansy
JK
,
Seeger
H
,
Rajab
TK
.
Impact of electronic messaging on the patient-physician interaction
.
J Telemed Telecare
2009
;
15
:
243
250
[PubMed]
30.
Liebhaber
AB
,
Grossman
JM
.
Physicians slow to adopt patient e-mail
.
Data Bull (Cent Stud Health Syst Change)
2006
:(
32
):
1
2
[PubMed]
31.
Kittler
AF
,
Carlson
GL
,
Harris
C
, et al
.
Primary care physician attitudes towards using a secure web-based portal designed to facilitate electronic communication with patients
.
Inform Prim Care
2004
;
12
:
129
138
[PubMed]
32.
Miller
DP
 Jr
,
Latulipe
C
,
Melius
KA
,
Quandt
SA
,
Arcury
TA
.
Primary care providers’ views of patient portals: interview study of perceived benefits and consequences
.
J Med Internet Res
2016
;
18
:
e8
[PubMed]
Readers may use this article as long as the work is properly cited, the use is educational and not for profit, and the work is not altered. More information is available at http://www.diabetesjournals.org/content/license.

Supplementary data