Comparison of the Usability, Accuracy, Preference, and Satisfaction of Three Once-Weekly Glucagon-Like Peptide 1 Receptor Agonist Pen Devices in People With Type 2 Diabetes

This was a multicenter, prospective cohort study. To evaluate study outcomes, participants completed an individual interview with a study investigator to model pen device demonstrations and complete usability and preference surveys. Demonstration pen devices and injection pillow foams were used. Participants were aware that they would not receive any subcutaneous injection nor any active drug component. Study participants were provided the remuneration of a $50 gift card for participating. A sample size calculation was performed based on our previous study (28), indicating that at least 40 participants were needed to show significance in the accuracy rating. We determined that 60 participants would allow for additional subgroup analysis regarding the secondary outcome. This study was approved by the Colorado Multiple Institutional Review Board.

The study population included individuals aged 18–89 years with type 2 diabetes who were naive to GLP-1 receptor agonist therapy. People with type 1 diabetes; non-English speakers; individuals with self-reported impairment of cognition, hearing, vision, or dexterity; and those employed in a position with a role in treating or educating people with diabetes (e.g., health care workers) were excluded.

Study participants were recruited between June 2019 and May 2020 from four diverse outpatient clinics throughout Colorado, including an endocrinology specialty clinic, a primary care patient-centered medical home, a women’s health primary care clinic, and a family medicine federally qualified health center. Participants were recruited primarily from populations receiving clinical care from the research investigators or under the care of a practice colleague at the investigators’ respective sites.

Potential participants were screened for eligibility at the time they presented for a routine clinic visit or were contacted via telephone in advance of a visit. Study recruits received notification and assurance that participation in the study had no effect on their access to or the quality of clinical care.

Three investigators performed the interviews. Study participants were individually interviewed by one of the investigators in a private room (e.g., clinic office space, exam room, or conference room) for ∼60–120 minutes. Interviews were audio-recorded to allow for review of participant responses at a later date, if needed, and participants were informed that they would be recorded before the interviews started. Each participant was given blank paper and a pencil to take notes throughout the interview, if desired.

To start the interview, study participants first signed an informed consent form and then completed a self-administered, written intake questionnaire that documented their eligibility for participation and demographic and background health information and assessed their baseline willingness to use injectable medications. For individuals deemed otherwise eligible for study inclusion, a health literacy assessment was verbally administered by the investigator using a widely used, validated screening tool (Newest Vital Sign [NVS]) (30).

Participants completed three cycles of pen device demonstrations that included 1) preparing for demonstration by watching the manufacturer’s instructional video of the product up to two times and reviewing printed instructions for use, 2) demonstration of the pen device without support from the observer or use of printed instructions, and 3) completion of written satisfaction and preference surveys. Participants were instructed to demonstrate and/or verbalize each step of the demonstration; for example, participants could choose to state that they would wash their hands instead of physically completing this task. The order of the pen device demonstrations was randomized using 3 × 3 Latin squares. The time taken to demonstrate the use of each device was tracked using a digital stopwatch.

The study investigators developed an accuracy checklist for each device based on the manufacturer’s instructions for use (Supplementary Appendix S1). Based on this checklist, investigators determined that dulaglutide and exenatide XR BCise each had nine required steps, and semaglutide had 11 required steps. Investigators tracked whether participants physically demonstrated or verbalized each step correctly or incorrectly or omitted the step. To ensure consistency in using the task evaluation form and to standardize ratings, investigators met at baseline and after one-third of participants (n = 20) had completed the study procedures.

After each device demonstration, participants completed a three-item satisfaction questionnaire and a six-item usability questionnaire. A seven-point Likert scale (with 1 being “very unsatisfied” and 7 being “very satisfied”) was used to assess satisfaction with each device and included questions regarding comfort of using the device, time needed to prepare the injection, and overall experience with the device. For usability, a five-point Likert scale (with 1 being “very difficult” and 5 being “very easy”) was used to assess ease or difficulty of various preparation and injection steps, understanding of device instructions for use, and overall rating.

For overall preferences, participants completed an eight-item questionnaire using a three-point ranking system with 1 indicating “most preferred” and 3 indicating “least preferred.” The ranking questionnaire was followed by an additional three questions designed to ascertain how likely the participants were to use injectable medications, how likely they were to use these three specific pen devices, and how likely they were to recommend those same three medications to other people. The latter three questions were ranked on a scale from 0 to 10, with 0 indicating “not at all” and 10 indicating “very likely.”

Differences in outcomes across pen devices were compared using univariate mixed models, adjusting for multiple comparisons using Tukey’s adjustment. The Wilcoxon signed rank test was used to look at changes within individuals for nonnormal data. To compare usability, ease, and satisfaction among the devices, we used generalized linear mixed models with random effects. Independent variables that were tested between groups included baseline willingness to use injectable medications, previous use of a non–GLP-1 receptor agonist injectable medication, and health literacy. Descriptive statistics were used for other analyses. A P value <0.05 was deemed significant. All analyses and plots for this project were generated using SAS, v. 9.4, software (SAS Institute, Cary, NC).

Sixty participants were included in the study, with a mean age of 61 years. Half of the participants were male, 65% were Caucasian, and 43.3% had previously used a non–GLP-1 receptor agonist injectable medication (Table 1). A majority of study participants (63.3%) had adequate health literacy based on the NVS screening.

Overall, a majority of participants preferred dulaglutide over semaglutide or exenatide XR BCise (74.6 vs. 13.6 and 11.9%, respectively), as shown in Figure 1. A majority of participants indicated that dulaglutide was the easiest pen device to learn how to use, yielded the best confidence of correct and complete dose delivery, had the best exterior design and styling, and was the easiest on which to set a dose. However, the semaglutide pen device was noted to have the best size and portability. Within this survey, two questions were negatively framed. Based on these negatively framed questions, exenatide XR BCise was noted as the pen device requiring the most effort to inject and was least likely to be recommended to others with type 2 diabetes.

Overall, user satisfaction with all three pen devices was high on average, with mean ratings of 5 or above on a seven-point scale, correlating to ratings of “somewhat satisfied” or above (Figure 2). Participants indicated greater satisfaction with the dulaglutide pen device compared with the exenatide XR BCise or semaglutide pen devices (P <0.01 for each), including comfort of use (P <0.01 for each) and time to prepare the injection (P <0.001 for each).

Participants rated the dulaglutide pen device easier to use than the exenatide XR BCise or semaglutide pen devices (P <0.001 for each), as shown in Figure 3. This trend was consistent for other aspects of use, including mixing the medication, removing the needle cap or cover, and knowing when the injection is complete. Injecting the medication was found to be easiest with the dulaglutide pen device (P <0.001 for each comparison) and was rated the same between the semaglutide and exenatide XR BCise devices. Understanding the instructions for use was easiest with dulaglutide (P <0.001 for each comparison) and was deemed similar between semaglutide and exenatide XR BCise (P = 0.89).

The mean rating for willingness to inject medications, on a scale of 0–10, was 7.35 (SD 2.98) at baseline and 7.68 (SD 3.24) at the completion of the study (P = 0.2251). Willingness to inject medications at both baseline and completion of the study did not differ between those who had previously used an injectable medication and those who were injection-naive.

Participants also rated on a 0–10 scale how likely they would be to use one of the once-weekly GLP-1 receptor agonists for management of their type 2 diabetes, with a mean rating of 7.93 (SD 3.05). The mean rating of likeliness of recommending one of the once-weekly GLP-1 receptor agonists to someone else with type 2 diabetes was slightly lower at 7.86 (SD 3.18).

Accuracy at demonstrating injection technique was lower when using the dulaglutide pen device compared with the exenatide XR BCise and semaglutide devices (P <0.001 for each), as shown in Table 2. Handwashing was the most commonly omitted step for all pen devices (73% with dulaglutide, 77% with exenatide XR BCise, and 75% with semaglutide). Other commonly omitted steps included explanation of proper storage such as taking the medication out of the refrigerator (77% with dulaglutide, 45% with exenatide XR BCise, and 75% with semaglutide) and ensuring that the medication does not appear cloudy (61% with dulaglutide, 85% with exenatide, and 58% with semaglutide). The most common errors in demonstrations of dulaglutide injection were related to making sure the pen device was locked before removing the gray base cap (12% incorrect, 7% omitted) and waiting for the second click after completing the injection (15% omitted). With the semaglutide pen device, the most common error was related to completing a flow check (28% incorrect, 18% omitted). Multiple errors were seen with demonstration of exenatide XR BCise injection, including the steps of shaking the auto-injector hard for at least 15 seconds until the medicine was mixed (10% incorrect, 3% omitted) and checking to see if the medication was mixed (17% omitted). Many participants did not wait 15 seconds before releasing the auto-injector (13% incorrect, 13% omitted). Three exenatide XR BCise pen demonstrations resulted in a broken pen device.

The demonstration time varied for each device. The dulaglutide device took less time to demonstrate compared with the exenatide XR BCise and semaglutide devices (P <0.001), as noted in Table 2. There was no significant difference in time to inject between exenatide XR BCise and semaglutide.

As a secondary analysis, the potential influence of health literacy on outcomes related to satisfaction, usability, preferences, accuracy, and timing of demonstration was assessed. Based on the NVS health literacy screening, 36.7% of participants had limited health literacy. Limited health literacy did not affect satisfaction and usability ratings; these ratings were not significantly different between those with adequate and limited health literacy for any pen device.

Health literacy also did not significantly affect the accuracy of injection demonstration. Participants with low health literacy took longer to demonstrate injection technique compared with those with adequate health literacy, but this difference was not significant.

GLP-1 receptor agonists are a preferred therapy in the treatment of type 2 diabetes and have multiple potential benefits. Limitations to their use often include prescriber concerns about a person’s willingness to use injectable therapies and the usability of the pen devices. However, we found that willingness to use an injectable, once-weekly GLP-1 receptor agonist was high, regardless of prior experience with non–GLP-1 receptor agonist injectable therapies. Participants found the pen devices easy to use overall and were satisfied with their features. Although all pen devices were generally accepted, most participants preferred the dulaglutide pen over the semaglutide and exenatide XR BCise pens. It is noteworthy that preference and accuracy were not significantly affected by prior experience with non–GLP-1 receptor agonist injectable therapy use or health literacy. Although not seen as an influencing factor in the results, participants with previous injection use frequently verbalized that the semaglutide pen device was more comfortable to use.

Accuracy was overall near 70%, and the most commonly missed steps were less clinically impactful (e.g., handwashing). There were erroneous demonstrations with each pen device that were more clinically impactful, and these examples could potentially lead to injection failure, inaccuracy of dose delivered, or injection of improperly mixed medication. This finding highlights the importance of direct education about pen device use and injection technique before initiation of a GLP-1 receptor agonist, as accuracy concerns remained after self-study of the product instructions and instructional video.

In consideration of educational methods, study investigators noted that most participants did not use the printed instructions for use and instead relied on education provided in the manufacturers’ instructional videos. There were differences in the training videos, including length (ranging from 3 to 8 minutes), key points emphasized, and format. Some participants noted that the training videos were distracting because they offered extraneous content, which could have limited participants’ ability to comprehend instructions. Additionally, most manufacturers only offered accessible training videos in one or two languages, which may limit their usability in diverse populations. This finding further emphasizes the need for individualized education and training, a need that could be filled by various health care providers, including community pharmacists at the time of dispensing a GLP-1 receptor agonist.

Preferences and perceptions play a key role in adherence, which in turn affects the real-world effectiveness of GLP-1 receptor agonists. Although clinical studies of these agents demonstrate their robust glucose-lowering efficacy, large claims database studies suggest less effect in real-world populations, with medication-taking behaviors accounting for much of the observed difference in A1C reduction in clinical practice versus randomized controlled trials (31,32). Thus, the American Diabetes Association’s Standards of Medical Care in Diabetes (33) recommend that treatment decisions be made collaboratively with people with diabetes based on their preferences. Shared decision-making is an important element in therapy selection. Based on this study, most participants preferred dulaglutide over semaglutide or exenatide XR BCise injections. However, participants commented on other factors that could have influenced the overall decision that were not individually assessed, such as fear related to ability to see the semaglutide needle and consideration of the multidose nature of semaglutide to limit plastic waste.

One of this study’s strengths was recruitment from four distinct practice settings across two different regions in the state of Colorado, including primary care, specialty, and underserved-focused clinics. This strategy allowed for a diverse group of participants from varying age, racial/ethnic, and socioeconomic groups. In addition, consistency was ensured among sites in a number of ways. Randomization of pen demonstration order limited learning bias. Additionally, the pen device training provided to each participant was consistent, as all subjects watched the same instructional videos and had access to the same printed instructions for use.

A few limitations must be acknowledged. The number of steps required for demonstration differed among the pen devices. Thus, accuracy ratings may be slightly skewed for pen devices that had fewer overall steps, where each step counted for a higher percentage in the accuracy score. In addition, this study used demonstration pen devices instead of actual injectable medications. Thus, one variable that could affect practical patient injection preference includes comfort of the injection itself. For example, needle size varies among the devices. The dulaglutide pen device contains a 29-gauge, 5-mm needle; the exenatide XR BCise pen uses a 23-gauge, ∼7-mm needle; and the semaglutide device uses a 32-gauge, 4-mm needle, which could technically be changed to a different size if a user so desired. Our study did not assess these differences that could be seen in real-world practice. Additionally, participants commented on a potential for biased responses based on the frequency of seeing certain products on television commercials.

Overall, the dulaglutide pen device was preferred over the semaglutide and exenatide XR BCise devices by a majority of participants, based on usability and satisfaction. Although the dulaglutide pen took the least amount of time to demonstrate, its associated demonstration accuracy was lower. This was the first study to compare participant preferences for the three available once-weekly GLP-1 receptor agonist injectable pen devices, and the results provide important information to contribute to shared decision-making when prescribing a GLP-1 receptor agonist. Additional studies may be needed if new GLP-1 receptor agonists and pen devices become available.

The Investigators acknowledge biostatistician Garth Wright of the University of Colorado Skaggs School of Pharmacy and Pharmaceutical Sciences for his contributions to statistical analyses, as well as Safeway pharmacist Alysa Campuzano; Kelli Rourke, an oncology pharmacy resident at Froedtert Hospital and the Medical College of Wisconsin; and Mackenzie Betancur, an ambulatory care pharmacy resident at the William S. Middleton Memorial Veterans Affairs Hospital.

This project was funded through an internal seed grant from the Department of Clinical Pharmacy at the University of Colorado Skaggs School of Pharmacy and Pharmaceutical Sciences.

No potential conflicts of interest relevant to this article were reported.

S.A.W. participating in conceptualization, methodology, validation, formal analysis, investigation, resources, data curation, writing, review and editing, visualization, supervision, and funding acquisition. M.P.S. participated in conceptualization, methodology, investigation, writing, review and editing, and visualization. K.K. participated in investigation, data curation, and writing. J.M.T. participated in conceptualization, methodology, investigation, resources, writing, review and editing, visualization, and project administration. S.A.W. and J.M.T. are the guarantors of this work and, as such, had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

This research was presented at the virtual American College of Clinical Pharmacology Annual Meeting, 19–30 October 2020, and published in abstract form for that purpose (https://doi.org/10.1002/jac5.1351)