SweetMama: Usability Assessment of a Novel Mobile Application Among Low-Income Pregnant People to Assist With Diabetes Management and Support

This study was a prospective, mixed-methods investigation of the usability of SweetMama for low-income pregnant people with GDM or type 2 diabetes. Its goals were to 1) confirm that the SweetMama app was functioning properly across a wide range of devices and operating systems, 2) provide information about real-world app usage and user satisfaction, 3) analyze participant patterns of use and experiences to inform future iterations, and 4) collect quality assurance data to support refinement for later longitudinal investigations.

Descriptions of SweetMama’s functionality and design process have been published (12,30,31). SweetMama is a patient-centered, interactive, web-based smartphone app designed to support diabetes self-care and education for low-income pregnant people. Its curriculum was curated by health care providers with experience treating pregnant people with diabetes. SweetMama sends users motivational, tip, and goal messages aimed at promoting self-efficacy each week and provides low–health literacy educational materials regarding diabetes and pregnancy. SweetMama features a library containing recipes, educational videos, links to local resources, and an extensive collection of health information obtained from both our clinic and external sources. SweetMama also provides users with a portal for technical help queries, information about the clinical site (e.g., our address), the user’s gestational age, and appointment reminders. Interactive features give users the ability to customize their library by marking content as “favorite,” connect in more depth with individual messages by clicking on links to external trusted resources, and request technical help. The recipe library additionally allows users to create a custom library of recipes based on their selection of favorites. In addition, a goal-setting component includes the interactive development of diabetes-related goals with the clinician. These goals then serve as a basis for goal reminders sent through the app.

Modifications were made to the curriculum in response to findings collected in earlier phases of work (12). These changes included extending the curriculum to continue through 6 weeks postpartum, enhancing visual features, using more professional punctuation, and including reminders to schedule future appointments.

Participants were recruited from two obstetrics and gynecology practices at Northwestern Memorial Hospital in Chicago, IL, between March and August 2019. Individuals were eligible for participation if they were pregnant or within 6 weeks postpartum, were ≥18 years of age, had been diagnosed with GDM or type 2 diabetes, spoke English as a first language, possessed a personal smartphone, and were considered low-income based on public insurance status or income assessment (<200% of the federal poverty guideline). Individuals with type 1 diabetes were excluded from the investigation because the SweetMama program was not designed for the unique and more complex educational, monitoring, and clinical needs of pregnant individuals with type 1 diabetes. The target sample size was set at 20 participants, which was considered sufficient for early-stage mHealth research to achieve usability testing goals (32–35). All participants provided written informed consent. All study activities were approved by the Northwestern University Institutional Review Board.

Upon entry, participants completed surveys on demographic information, mobile device usage, and Internet access. Participants also completed assessments of health literacy and patient activation, which have been reported previously (36). They were oriented to SweetMama by a research assistant and via a brief in-app video describing key elements, including how to request technical assistance. Participants were instructed that SweetMama was not a substitute for clinical care and that the technical help function was not for clinical queries. Individuals were instructed to use SweetMama at their own discretion for 2 weeks.

During each participant’s 2-week enrollment, delivery of the SweetMama curriculum was customized based on gestational age and therapeutic plan (insulin versus no insulin), allowing for variation in delivered messaging content to test the SweetMama curriculum schedule and algorithm. Although participants received curriculum messages specific to their gestational age, they could access all other features. After 1 week of SweetMama use, participants were contacted via phone for a technical check-in to ensure proper app function and resolve any technological malfunctions.

After 2 weeks, participants completed in-person assessments of their experience with SweetMama. Surveys included the System Usability Scale (SUS) (37,38), which is used to assess satisfaction with and usability of a product, and the Usefulness, Satisfaction, and Ease of Use (USE) Questionnaire (39), which evaluates user interfaces across domains of usability, satisfaction, and perceived ease of use. Participants additionally completed a 30-minute in-depth, semistructured interview with a research assistant. Framed by the Health Belief Model (HBM), this interview inquired about patients’ experiences with SweetMama, its ease of use, and their satisfaction and perceptions of overall usefulness of the app (40). Interviews were digitally recorded and professionally transcribed. Participants received gift cards upon enrollment and study completion.

Interview transcripts were uploaded to Dedoose, a secure, online qualitative data-organizing platform (41). Transcripts were analyzed using the constant comparative method (42,43). Independent coders reviewed a subset of five transcripts and identified recurring concepts called subthemes, which were organized into a preliminary codebook. This codebook was discussed by all team members, refined to ensure consistency and clarity, and subsequently used to iteratively code the remaining transcripts by two independent coders. The codebook was modified at frequent meetings of the research team. Discrepancies between coders were resolved on a code-by-code basis. Subthemes were eventually organized into overarching themes focused on users’ SweetMama experiences.

User interaction data were collected after 14 days of enrollment, including data regarding the frequency and duration of SweetMama user interactions, navigations to each static and dynamic feature, and use of the “favorites” function. Only data related to how and when SweetMama was used were collected, and the format in which data were stored contained no identifying information about users.

Descriptive statistics were calculated for all variables of interest. Categorical variables were summarized with counts and percentages, and continuous variables with means and standard deviations or medians and interquartile ranges, as appropriate. Pie and bar charts were used to illustrate the distribution of usability and satisfaction ratings.

During the study period, 24 individuals enrolled in usability testing. Participants largely identified as non-Hispanic Black, and 50% had GDM (Table 1). The majority (n = 23) were pregnant, and the median gestational age at enrollment was 27 weeks (range 12–37). Of the 24 participants, 23 individuals used SweetMama and provided user interaction data, whereas one did not open any messages and was considered a nonuser. Of the 23 users, 22 provided exit interview data.

Analysis of qualitative data yielded four categories of participants’ experiences with SweetMama: 1) SweetMama design and technical strengths, 2) beneficial effects of SweetMama use on self-management, 3) limitations of current SweetMama experience, and 4) desires for SweetMama expansion. Themes and associated subthemes are described below and summarized in Tables 2–5, followed by user interaction and usability results.

The first category featured comments regarding SweetMama’s design and technical strengths (Table 2). Many participants commented on the strength of the concept underlying SweetMama—that the app filled the gap for pregnant people with diabetes. Multiple participants recognized the lack of available pregnancy apps that catered specifically to people with diabetes. Some described SweetMama as being their main or only source of diabetes-related information. One participant stated, “. . . [I]f I have the SweetMama app and it’s giving me the information that I need, I don’t think I need another app.”

Users perceived SweetMama’s interface to be visually appealing. With a colorful logo, a personalized greeting, up-to-date gestational age, and appointment reminders, the homepage was described as “eye-catching” and containing “all [of the patient’s] information.” Many participants viewed the color scheme favorably and appreciated the straightforward way in which information was presented. One participant said, “If it talks to you about a topic, it gives you right away the main information—[in the] first paragraph, I will see it. So it’s mainly directly clear [and] simple.”

Participants commented on SweetMama’s usability, saying the application was intuitive with minimal navigation required to retrieve desired information. “[SweetMama] was self-explanatory,” one participant said. “You click messages, you see them, [go to the] library, and then it actually told you, and you could look at the recipes. Others mirrored this sentiment, stating that “a child can navigate [SweetMama] pretty well” and that SweetMama was a “very easy tool.” One highly praised aspect of SweetMama’s architecture was the “favorite” feature, which allows a user to mark as “favorite” preferred resources to easily access them later. Users reported frequently using this feature to bookmark their favorite recipes, goals, and videos. Notably, participants commented that few, if any, technical problems existed within SweetMama.

Participants indicated that SweetMama had beneficial effects on their diabetes self-management (Table 3). One valued feature was the goal-setting component, in which participants set diabetes-related goals with their health care provider; goals were subsequently uploaded and delivered as a goal query message later that week. Participants reported that this goal-setting exercise encouraged self-reflection and self-awareness. By discussing and being reminded of goals, participants felt accountable for their health-related benchmarks (Table 3).

A cornerstone of the SweetMama app is its regular delivery of motivational content. Participants reported that this motivational curriculum promoted self-regulation and provided encouragement. One participant said these tips and encouraging messages “help motivate you day by day because this is, for some people like me, work in progress.” Other participants recalled specific instances with motivational messages (Table 3).

SweetMama also provides users with one to two practical tips per week and displays reminders of upcoming prenatal appointments on the home screen. This content was generally regarded as providing helpful cues for self-management and clinical engagement. For example, one participant explained that SweetMama reminded her to perform diabetes tasks: “When they tell you the little messages that says, ‘Oh, remember to walk’ or ‘Remember to eat this’ or ‘Remember not to eat too much of that.’ ‘Keep track of your glucose’ so many times a day. That’s good reminders.” Other cues prompted participants to take their prenatal vitamins, exercise, and use their electronic health record to ask questions, which often prompted ongoing clinical engagement (Table 3).

Nearly every participant positively described the utility of the extensive library of health resources, recipes, and social resources within SweetMama. The plethora of resources available in the library was viewed as a centralized repository of local and digital health resources that eased the burden of locating information (Table 3). Additionally, the SweetMama library contains geographically specific resources, including links to city social services, locations of food pantries and farmer’s markets, and registration for low-cost postpartum classes. One participant said, “[The library is how] I found out about the places like at Humboldt Park where they have other resources and stuff like that.” The most widely appreciated feature of the SweetMama library was the recipe repository, which contains diabetes-friendly recipes selected by a team of physicians, nurses, dietitians, and health educators. Participants found this recipe library to be helpful and reported that it often encouraged healthy eating. One participant commented on the cultural relevance of the recipes she found, saying, “It has helped. Just going through the recipes. I didn’t know you could eat certain foods. Being Hispanic, we love hominy soup, and I found the recipe there for hominy soup.” Another commented on the long-term utility of the recipes: “I bought things that I liked and I got from the app [. . . ], which was really beneficial, and I think it’s something I’m going to eat after I’m pregnant.”

Participants also found that the library materials promoted ongoing education about diabetes and nutrition. Many resources gave practical and healthy ingredient substitutes, explained carbohydrate counting, and counseled on portion sizes. Numerous participants reported that SweetMama helped them better understand their diabetes. For example, “I think it has the proper information, like about diet, portions, what to eat, what not to eat. Not big changes. But I think providing me information every day is, it’s important.” Being able to reference vetted health information was viewed as helpful (Table 3).

Despite largely positive reviews, participants identified two main categories of SweetMama limitations: the desire for more colors, pictures, and videos, and limitations of Internet-based mobile apps (Table 4).

Although many participants liked the basic SweetMama color scheme, many also desired more colors—a request often cited in conjunction with a desire for more graphics, pictures, and videos. Users desired images of the finished recipes from the library to be displayed on the SweetMama page (as opposed to on the external landing page, where the recipe was digitally housed). They also desired videos (e.g., videos of their clinical team demonstrating the proper technique for insulin administration).

Other shortcomings included those associated with a Web-based mobile app. Many participants believed the current text-to-log-in process—a security feature—was cumbersome and disliked toggling between text messages and the SweetMama interface. Although few participants experienced problems opening SweetMama resources, participants disliked clicking on links to view them on an Internet browser and desired a way to view resources within the app.

Participants also voiced their desire for additional features and improvements (Table 5). The most ubiquitous desire was for a platform for peer-to-peer support. Many participants expressed fear or confusion at their diabetes diagnosis and felt that success stories from peers would provide them with additional motivation and reassurance. As one participant said:

Participants envisioned peer-to-peer support in the form of chat rooms, blog posts, social media–like feeds, or pre-recorded podcast episodes of shared success stories.

Many users desired the ability to tailor features within SweetMama. This included the desire to customize notification settings, upload a photo of the user or create an avatar in lieu of the SweetMama logo, or change the default language. Participants desired the ability to set their own goals—which were not always diabetes-related—in the app, saying, “Maybe, like, once she’s [baby] 1 [year old], maybe get her off the bottle, or stuff like that.”

Another common request was for increased health maintenance support. Although the library contains 240 different resources related to GDM, type 2 diabetes, nutrition, postpartum concerns, recipe support, and city-specific resources, users desired even more extensive resources related to diabetes-specific health maintenance, including weight, glucose, and activity-tracking features. One participant requested specific nutritional advice, saying:

Others desired more information in the messages. Although tip messages were intentionally brief (with additional detail offered in links), some desired more content in the messages themselves. One participant said, “You know, more tips, because the tips weren’t . . . I’m not gonna say they weren’t really good, but they’re very short, you know? The tips are more like a goal than an actual tip.”

Over 2 weeks, users opened SweetMama a median of eight times (Table 6). The median duration of use per episode was 1 minute, and the median total duration of use over 2 weeks was 20.5 minutes, ranging from 4.0 to 66.0 minutes.

Users interacted with every static (i.e., not responsive to user input) feature on the SweetMama platform, including the home screen, “about us” section, message center, and library. Users most frequently navigated to the message center (median 20.5 navigations per participant), where the tip, goal, and motivational messages are housed. Participants also frequently navigated to the dynamic features (i.e., those that offer users the ability to connect with more information and that respond to user input). Users performed a median of 2.5–3.0 navigations to each type of dynamic feature during the study period (Table 6).

Findings from usability surveys corroborated the user interaction metrics. SUS results indicated that 13 participants (59%) rated SweetMama in the highest category of usability (Figure 1A). Another 32% of participants rated SweetMama as having marginal usability, suggesting that overall usability metrics were high for an early-stage prototype. Data from the USE Questionnaire showed that participants rated SweetMama ∼6 out of 7 in categories of usefulness, ease of use, ease of learning, and satisfaction (Figure 1B).

GDM and type 2 diabetes during pregnancy are growing public health problems that disproportionately affect medically underserved individuals. mHealth technology may be a promising avenue for delivering educational and motivational content to pregnant people, particularly those with greater barriers to care (13). We designed a motivational and educational mHealth app called SweetMama for low-income pregnant individuals with diabetes and assessed its usability in a 2-week trial.

SweetMama users found it to be visually attractive, relatively easy to navigate, culturally tailored, and functional, with few technical problems. These positive design and functionality reviews were also reflected in high usability and ease-of-use scores. SweetMama was described as filling a gap as a diabetes-specific pregnancy app. It positively affected participants’ ability to self-manage their diabetes by encouraging goal-oriented behaviors, providing regular reminders for diabetes-oriented health management tasks, offering motivation, and serving as an educational and practical resource. Participants desired more images and videos and wished SweetMama was a more streamlined, standalone tool. Additionally, participants desired more customizability, peer-to-peer support, greater information provision, and support with diabetes health maintenance tasks. Each of the areas for improved usability is being considered for future iterations in longer-term feasibility testing. For example, in response to usability assessments, future iterations will include instructional videos, easier-to-navigate buttons, and streamlined visual features to reduce complexity.

Prior data indicate that interventions with theoretical underpinnings are most likely to deliver the greatest benefit to users (26). Thus, SweetMama was developed using the HBM and the Theory of Self-Efficacy. The HBM explains individuals’ engagement in health behaviors via perceived disease susceptibility and severity, as well as barriers to and benefits of taking action, which together predict the likelihood that a person will adopt a behavior (40). The Theory of Self-Efficacy focuses on individuals’ belief in their own ability to successfully perform a behavior (44). These theories shaped the periodic delivery of SweetMama content, which was designed to provide brief but frequent doses of motivation and education. User interaction data indicate that users opened SweetMama, on average, eight times in 2 weeks and used the app for an average of 1 minute per episode. These results are consistent with our expectations of SweetMama use and are promising findings, as support accessed in short, frequent doses is commonly thought to be a successful form of intervention (45).

Despite participants’ requests for health maintenance trackers (e.g., blood glucose and food logs), we opted not to create such functions within SweetMama. Because smartphone-linked glucose meters and app-based food logs have become routine elements of perinatal diabetes care and were available to all participants, we believe that creating tracking capabilities within SweetMama would be redundant and misaligned with the app’s goal of providing regular educational and motivational content without requiring clinicians to be monitoring this service in the background. Given the easy access to these technologies and to patient portals (e.g., Epic MyChart), SweetMama was designed to interface alongside these technologies, allowing goal-setting and educational content to be delivered in response to reviews of clinical data by clinicians. In addition, the tracking of personal health information would also require SweetMama to have advanced security and connectivity features, which would be resource-intensive and reduce the app’s scalability. However, user perspectives on these health maintenance features have emphasized the importance of connectivity of tools such as SweetMama with other tools that individuals may regularly use. Future work may consider how to integrate novel mHealth tools with commercially available apps, electronic health record portals, or smart glucose meters.

This user-centered design study has a number of strengths, particularly the inclusion of end-users in evaluative stages. Because SweetMama is designed for low-income, minority individuals, inclusion of such individuals is an essential element of design. Furthermore, iterative modifications based on user input allowed for ongoing feedback on SweetMama, further increasing its applicability to the target population of individuals with GDM or type 2 diabetes.

Our study is limited by inclusion of only English-speaking participants from a single academic center, which may limit the range of perspectives and the generalizability of findings. In addition, SweetMama was designed for and tested in a low-income population to address gaps in health care and health behavior support that are known to exist for this population. Thus, the app itself may have limited generalizability to other populations. The app also was in an early phase of development and thus existed as a Web-based smartphone tool, which limited some functionality. Furthermore, participants used SweetMama for 14 days; this duration may have limited the depth of exposure to all SweetMama features and to the app’s extensive library. However, the goal of this phase was not to assess long-term engagement or clinical outcomes, which we recognize are important next steps. Future work will investigate the appropriate dose of SweetMama use to influence perinatal outcomes, differences in patient characteristics (e.g., diabetes type or pregnancy versus postpartum status) associated with SweetMama use, the effect of SweetMama use on glycemic control during pregnancy, and the effect of SweetMama exposure on both maternal and neonatal clinical outcomes.

In summary, low-income pregnant people with GDM or type 2 diabetes who used the SweetMama app found it to be user-friendly, informative, and engaging. Users provided essential feedback to support the next phase of study. SweetMama development will focus on incorporating improvements suggested in this phase such as including educational videos, incorporating more graphics, and considering options for peer support. Future work must study the feasibility of using the app over the full pregnancy and postpartum period, as well as its effectiveness at improving diabetes-related perinatal outcomes.

This work was supported by the National Institute of Child Health and Human Development (R21 HD094271) and the Friends of Prentice FY2019 Women’s Health Grants Initiative.

No potential conflicts of interest relevant to this article were reported.

L.M.Y. designed the study, obtained funding, supervised the conduct of the trial, analyzed and interpreted the data, wrote the manuscript, and contributed to the overall study supervision. K.L. and J.J. conducted the trial, interpreted the data, and drafted the manuscript. C.N. provided trial supervision, contributed to the discussion, interpreted the data, and reviewed/edited the manuscript. R.S. designed the intervention, generated the data, interpreted the data, and reviewed/edited the manuscript. C.Y. analyzed the data, interpreted the data, and reviewed/edited the manuscript. M.A.S. supported the study design and funding, contributed to the discussion, and reviewed/edited the manuscript. L.M.Y. is the guarantor of this work and, as such, had full access to all data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

This work was presented in abstract form at the 40th annual meeting of the Society for Maternal-Fetal Medicine in Grapevine, TX, 3–8 February 2020.