This study aimed to assess diabetes health information found on TikTok and quantify misinformation on TikTok. The authors assessed 171 videos through two health literacy tools, DISCERN and the Patient Education Materials Assessment Tool for Audiovisual Materials, to rate the understandability and actionability of online medical content. The findings from this study encourage health care professionals to use social media platforms to provide factual information about diabetes and advise online health care consumers to use reputable sources such as trusted diabetes organizations’ social media accounts, which tend to validate content with clinicians.
Young adults with type 1 diabetes often turn to social media networks for educational information. The diabetes online community (DOC) is a vibrant and diverse virtual space that spans several social media platforms, including Instagram, Facebook, Twitter, Reddit, and TikTok (1). Online patient communities provide a sense of belonging and offer psychosocial support through anecdotal relatability (1,2). Individuals use the DOC to share experiences related to blood glucose management, post pictures of continuous glucose monitoring systems, and even crowdfund the cost of insulin through sites such as GoFundMe (3). The DOC is known to have an overall positive impact on individuals with type 1 diabetes by improving their happiness and level of engagement and providing psychosocial support outside of medical centers (2,4). Young adults have significant lapses in diabetes care, with suboptimal glycemic outcomes (5–8). For this reason, the availability of quality diabetes education content is extremely important for this population.
TikTok, previously known as Musical.ly, is a social media platform on which users post short videos ranging from 15 seconds to 10 minutes in length (9). To date, TikTok is one of the most downloaded social media applications, with >1 billion users, and is popular among young adults 18–30 years of age (9–11).
In this age of digital globalization, social media platforms such as TikTok are considered by many to do more good than harm by creating supportive communities and efficiently translating health information to the public (12). For example, TikTok is one of the most popular platforms for promoting mental health–related reforms and information (13). However, TikTok can also promote and popularize harmful substances such as JUUL pod e-cigarettes and alcoholic beverages among adolescents and young adults (14). Importantly, sensitive and harmful ideologies are coupled with the spread of misinformation on the platform (9). TikTok’s #diabetes topic identifier has had more than 16 billion views, with 38% of the audience 18–24 years of age. The DOC uses TikTok to relay diabetes-related information and uses hashtag topic identifiers to garner a larger audience. The power of diabetes TikTok messaging was recently observed with drug shortages for semaglutide after posts about weight loss went viral, with #ozempic garnering more than 275 million views (15).
As more people are seeking health information on TikTok, creators are creating more content, resulting in a truly bidirectional relationship. There are a myriad of content creators whose prime focus is promoting diabetes health information (16). Content creators can be people with diabetes, health coaches, health care professionals (HCPs), and people without diabetes (16).
Despite the availability of important health information on TikTok, some advice on the site is misleading and even legally questionable (17–19). Most TikTok videos are not monitored or fact-checked, which might lead young adults—the vast majority of TikTok users—to mimic harmful behaviors (16). Despite the amount of content, number of users, and frequency of usage, there is a dearth of information about who is creating the content, the quality of said content, and whether the content influences health behavior. Despite the popularity of TikTok, the majority of diabetes health information shared on TikTok has not been assessed for accuracy (14,16).
Primary users of TikTok who consume diabetes-related information comprise a younger demographic who are vulnerable to receiving misinformation. To promote accurate, safe diabetes education, further research is needed on the quality of the diabetes information currently available on TikTok. In light of the DOC gaining traction on TikTok, this study aims to assess diabetes-related information found on the platform.
Research Design and Methods
Ethics Approval
This study was found to be exempt from institutional review board oversight by Ohio University and University Hospitals Cleveland Medical Center.
Data Collection and Extraction
First, popular diabetes-related hashtags were selected through the search algorithm. The five most popular hashtags included #t1d, #t1diabetes, #type1diabetes, #t1dtips, and #t1dcoach. On 29 June 2023, Apify web scraping software was used to download video content information. Inclusion criteria for the videos were that they were posted by a public account between 29 June 2022 and 29 June 2023. Videos posted before 29 June 2022 and duplicate videos were excluded. An initial scrape resulted in 173 downloaded videos based on the inclusion criteria. Two of these videos did not work, which led to a final sample size of 171 videos. The two videos had been taken down by their creator at the time of data analysis. The descriptive characteristics of each video (i.e., number of likes, views, and comments) were also captured through Apify. Figure 1 provides additional information on the study methods.
Video Assessment
REDCap (Research Electronic Data Capture), a data collection and management platform, was used to build a video quality assessment survey (20). All seven research team members individually assessed and coded the videos. If there were concerns regarding the assessment, team members consulted each other to reach consensus. Categorization criteria included the type of video creator, including health coaches, HCPs, people with type 1 diabetes, caregivers/parents of people with type 1 diabetes, noncaregiver family members of people with type 1 diabetes, individuals with no relation to diabetes mentioned, and other miscellaneous creators.
After the videos were categorized, the DISCERN instrument was used to assess the quality of video content (21). DISCERN is a tool most often used to evaluate health information. The DISCERN tool has been used in several other studies to evaluate online health information on TikTok and other media platforms (16,22–24).
The DISCERN measure includes 16 questions to assess information reliability (questions 1–8), the quality of information about treatment (questions 9–15), and the overall quality of the material (question 16) (25). Reliability of the information includes the aim of the content, the source of information, and its relevance. The quality of the information about the treatment evaluates whether the video explains how the treatment works, its risks and benefits, and additional choices for self-management. The final question rates the overall quality of the video as a source of information about self-management or treatment options. The first 15 DISCERN questions are rated with a Likert scale ranging from 1 = definite no to 5 = definite yes. Scoring for question 16 is independent of the first 15 items. The total score ranges from a minimum of 16 to a maximum of 80. Based on previous research studies, scores of 16–29 on this question are considered very poor quality, 30–40 indicates poor quality, 41–51 equates to fair quality, 52–63 represents good quality, and ≥64 indicates excellent quality (25–27).
The videos were also assessed using the Patient Education Materials Assessment Tool for Audiovisual Materials (PEMAT-A/V), a 10-question health literacy tool used to rate audiovisual and printed online materials based on understandability and actionability and to help determine whether patients will be able to understand and act on the information they receive (28). The PEMAT-A/V provides a score for understandability and a separate score for actionability. PEMAT-A/V has been used in several other studies to evaluate online medical information (24,29,30). For scoring, 1 = agree and 0 = disagree. The final understandability score is calculated by adding the scores for the understandability items and dividing the sum by the total possible points. This score was multiplied by 100 to create a percentage. Higher total scores represent coherent quality health education with actionable items.
Results
Descriptive Statistics
A sample size of 171 videos from 63 different content creators were extracted. SPSS, v. 29.0.1.0, statistical software was used to conduct a bivariate correlation analysis. The 171 videos used for this analysis were posted between 14 July 2022 and 29 June 2023. Videos meeting the inclusion criteria had a median view count of 263,400 (interquartile range [IQR] 126–123,400,000), a share count of 32 (IQR 0–27,300), and a comment count of 78 (IQR 0–29,400). The median video length was 38 seconds (IQR 3–239 seconds).
Video Creators and Content Types
From our sample, 45.6% were posted by health coaches, 39.8% by people with type 1 diabetes, 11.1% by caregivers of people with type 1 diabetes, 1.2% by HCPs, 1.2% by people with no relation to diabetes, 0.6% by noncaregiver family members of people with type 1 diabetes, and 0.6% by other (a caregiver who was also a health coach). Additional information is provided in Table 1.
Metrics of Videos and Content Creators
| Characteristics | Median (IQR) |
|---|---|
| Videos (n = 171) | |
| View count | 263,400 (126–123,400,000) |
| Share count | 32 (0–27,300) |
| Comment count | 78 (0–29,400) |
| Video length, seconds | 38 (3–239) |
| Creators (n = 63) | |
| Number of followers | 16,100 (28–14,200,000) |
| Number of likes | 887,600 (198–516,600,000) |
| Number of videos posted | 267 (2–4,357) |
| Characteristics | Median (IQR) |
|---|---|
| Videos (n = 171) | |
| View count | 263,400 (126–123,400,000) |
| Share count | 32 (0–27,300) |
| Comment count | 78 (0–29,400) |
| Video length, seconds | 38 (3–239) |
| Creators (n = 63) | |
| Number of followers | 16,100 (28–14,200,000) |
| Number of likes | 887,600 (198–516,600,000) |
| Number of videos posted | 267 (2–4,357) |
Video content was sorted into the following categories: useful tips, tricks, or information (37.4%); personal anecdotes (33.3%); comedy/humor (18.1%); and misleading/misinformation (11.1%). Details are provided in Table 2.
Descriptions of Videos and Types of Content Creators per Video (n = 171)
| Category | n (%) |
|---|---|
| Video content | |
| Useful tips, tricks, or information | 64 (37.4) |
| Personal anecdotes | 57 (33.3) |
| Comedy/humor | 31 (18.1) |
| Misleading/misinformation | 19 (11.1) |
| Content creator per video | |
| Health coach | 78 (45.6) |
| Person with type 1 diabetes | 68 (39.8) |
| Caregiver of person with type 1 diabetes | 19 (11.1) |
| HCP | 2 (1.2) |
| Person with no relation to diabetes | 2 (1.2) |
| Noncaregiver family member of person with type 1 diabetes (sibling) | 1 (0.6) |
| Other (caregiver who was also a health coach) | 1 (0.6) |
| Category | n (%) |
|---|---|
| Video content | |
| Useful tips, tricks, or information | 64 (37.4) |
| Personal anecdotes | 57 (33.3) |
| Comedy/humor | 31 (18.1) |
| Misleading/misinformation | 19 (11.1) |
| Content creator per video | |
| Health coach | 78 (45.6) |
| Person with type 1 diabetes | 68 (39.8) |
| Caregiver of person with type 1 diabetes | 19 (11.1) |
| HCP | 2 (1.2) |
| Person with no relation to diabetes | 2 (1.2) |
| Noncaregiver family member of person with type 1 diabetes (sibling) | 1 (0.6) |
| Other (caregiver who was also a health coach) | 1 (0.6) |
Three one-way ANOVA tests were conducted to determine whether levels of quality of health information (DISCERN score), understandability (PEMAT-A/V understandability score), and actionability (PEMAT-A/V actionability score) differed across content creator type. Six videos were excluded because their creator groups were too small for statistical analysis. Content creators were classified into three groups: health coaches (n = 78), people with type 1 diabetes (n = 68), and caregivers of people with type 1 diabetes (n = 19). Excluded cases were from the HCP group (n = 2), the group of noncaregiver family members of people with type 1 diabetes (n = 1), the group of creators with no relation to diabetes (n = 3), and the other group (n = 1) because of small sample sizes for analysis. Thus, a total of 165 videos were included in the analysis. Table 3 provides more information on video coring.
Video Scoring Descriptives (n = 171)
| Measure | Mean ± SD |
|---|---|
| DISCERN score | 24.33 ± 7.26 |
| PEMAT-A/V understandability score | 58.40 ± 27.88 |
| PEMAT-A/V actionability score | 38.79 ± 33.67 |
| Measure | Mean ± SD |
|---|---|
| DISCERN score | 24.33 ± 7.26 |
| PEMAT-A/V understandability score | 58.40 ± 27.88 |
| PEMAT-A/V actionability score | 38.79 ± 33.67 |
For quality of health information, there was a statistically significant difference between groups as determined by one-way ANOVA [F(2, 162) = 4.44, P <0.013]. The effect size, calculated as η2, was 0.05, indicating a small to medium effect. A Tukey post hoc test revealed that health coaches (P = 0.01) and people with type 1 diabetes (P = 0.02) had significantly lower scores for quality of health information than did the type 1 diabetes caregiver group. There were no differences in quality of health information between the content posted by health coaches and the content posted by people with type 1 diabetes (P = 0.927).
For understandability of health information, there was a statistically significant difference between groups as determined by one-way ANOVA [F(2, 162) = 13.59, P <0.001]. The effect size, calculated as η2, was 0.14, indicating a large effect. A Tukey post hoc test showed that content posted by people with type 1 diabetes and by caregivers of people with type 1 diabetes had significantly higher understandability scores than content posted by health coaches (P <0.001). The understandability scores for people living with type 1 diabetes and for caregivers did not differ (P = 0.098).
For actionability of health information, there was a statistically significant difference between groups as determined by one-way ANOVA [F(2, 162) = 8.26, P <0.001]. The effect size, calculated as η2, was 0.09, indicating a medium effect. Post hoc tests revealed that content posted by health coaches (P <0.001) and by people with type 1 diabetes (P = 0.002) had significantly lower actionability scores than content posted by caregivers of people with type 1 diabetes. There was no difference in actionability scores for content posted by health coaches and content posted by people with type 1 diabetes (P = 0.644). The sample size of HCPs was much smaller than we expected, contrary to other analyses of social media for health information.
Discussion
Principal Findings
This study’s principal finding was that people with type 1 diabetes along with caregivers of individuals with type 1 diabetes had significantly higher scores for understandability of health information than did health coaches (P <0.001). Our study also found that there was low understandability, actionability, reliability, and quality of videos found within the DOC on TikTok. Some unexpected results included a low number of videos categorized as misleading. The DOC usually aims to foster support among individuals with type 1 diabetes (4). This is one of the first studies to assess the quality of type 1 diabetes information on TikTok. Previous research focused on type 2 diabetes instead of type 1 diabetes (16). Our results align with previous TikTok studies on scoliosis, attention deficit and hyperactivity disorder, and oral health (22–24).
During the pandemic, reliance on social media for information-sharing was highly evident. The number of TikTok monthly users rose from 507 million in December 2019 to 1 billion globally in September 2021 (16,31). The skyrocketing number of TikTok subscribers can be attributed to people seeking out entertainment and information in times of isolation (9). Several studies have also found that people seek out social media platforms as a result of gratification theory, a mass communications paradigm that explains motivation toward internet engagement (12,32,33). In terms of TikTok, users are predominantly adolescents (9–12). These adolescent users can overconsume TikTok content because of the short lengths of its videos and are extremely susceptible to misinformation (34). Currently, there is unvalidated online information coupled with overconsumption of content, so it is important to have reliable sources of information from certified HCPs (9,34).
In our sample, HCPs made up a very small percentage of content creators, which is concerning with regard to young adult education. Online information dissemination continues to be a prevalent public health concern. Health and wellness coaches have gained popularity in recent years. Previously, health coaches connected mostly with adults through workplace wellness initiatives. Although personalized coaching sounds promising, health coaching is widely unregulated and unstandardized. Health coaching requires no certification and has no standards of care across coaches. During the course of our study, we found a wide variety of uncertified health coaches on TikTok, which is concerning given the nature of type 1 diabetes management.
Limitations
Video assessment was conducted by a group of seven interdisciplinary HCPs, scientists, and students to eliminate bias. Second, this study only evaluated videos from five hashtags. The hashtags focused on capturing the most popular type 1 diabetes videos and health coach tips. There are several hundred other diabetes-related hashtags. Additionally, several creators may not use hashtags, which makes our sample size specific to creators who promote content with hashtags. Future research can be conducted to assess videos under other popular diabetes hashtags. Finally, the DISCERN and PEMAT-A/V tools were used to assess the material. Using other tools such as the Journal of the American Medical Association benchmark criteria, a four-point scale that measures reliability and accuracy, might have led to different results.
Conclusion
Because of its low reliability, quality, understandability, and actionability scores, TikTok users should be wary of diabetes health information found on the platform. Our study was undertaken as a call for certified HCPs to use social media platforms. Although HCPs might be hesitant to use social media, it is imperative to note that this is a cost-free resource that can be used to provide comprehensive medical information. This study is also a warning for online health care information consumers to use reputable sources such as the social media accounts of trusted diabetes organizations, which tend to validate their content with clinicians. We also recommend clinical caution in the use of current online health coaching content. Future studies should include comparisons of the quality of health content from health coach creators and from HCPs. Future studies should also examine content created by individuals with diabetes and compare its quality to content from HCPs for clinical validation.
Acknowledgment
The authors thank Ohio University Heritage College of Osteopathic Medicine for providing opportunities for undergraduate summer research.
Funding
REDCap was funded through University Hospitals grant UL1TR002548. S.V. was funded by a grant from the National Institute of Diabetes and Digestive and Kidney Diseases Diabetes Institute Summer Interprofessional Research Experience for Undergraduates (1R25DK122952-01A1). A.K. receives undergraduate funding from the Case Alumni Association at Case Western Reserve University. J.E.B. receives funding from the Leona M. and Harry B. Helmsley Charitable Trust (G-2305-05992). J.R. receives predoctoral funding from the Dean’s Legacy Fellowship at Frances Payne Bolton School of Nursing.
Duality of Interest
No potential conflicts of interest relevant to this article were reported.
Author Contributions
All authors contributed to writing, reviewing, and editing the manuscript. S.V. contributed to the study conceptualization, investigation, and data curation. A.K. contributed to the investigation and data curation. J.E.B. contributed to the study conceptualization, investigation, data curation, formal analysis, and project supervision. J.R. contributed to the study conceptualization and investigation. A.S.H. was responsible for the study conceptualization, investigation, data curation, and project supervision. A.S.H. is the guarantor of this work and, as such, had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
