Cut Points for Identifying Clinically Significant Diabetes Distress in Adolescents With Type 1 Diabetes Using the PAID-T: Results From Diabetes MILES Youth–Australia

Many people living with diabetes feel burdened by the ongoing challenge of managing their condition and experience periods of frustration, anger, fear and helplessness related to their diabetes, which is referred to collectively as “diabetes distress” (DD) (1). For adolescents with type 1 diabetes, there are also various diabetes-specific issues related to age and stage of life (e.g., family conflict, parental overvigilance, and uncertainties about body image) (2). Approximately one-third of adolescents may experience elevated DD, although the prevalence is uncertain owing to the absence of a clinically valid cut point on adolescent-specific DD measures (3).

A threshold to indicate elevated DD enables the comparison of prevalence across populations for research purposes, the detection of people in clinical practice for whom intervention is needed to alleviate distress, and enables evaluation of the effectiveness of interventions. A standardized score ≥40 (range 0–100) on the Problem Areas in Diabetes (PAID) scale (derived from the mean +1 SD [4]) has been widely adopted as the threshold for elevated DD among adults, yielding prevalence rates of 19 to 31% in adults (5–7). It has also yielded a prevalence rate of 17% among adolescents, despite the measure not being designed specifically for this population (8). Yet, this cut point has not been validated clinically.

Given the subjective nature of DD, there is no gold standard measure against which to validate elevated DD. Therefore, other empirical approaches are required to establish a cut point. In 1978 Rose and Barker (9) outlined four approaches to determine a cut point on a continuous scale based on the intended purpose and available data:

1. Statistical significance, which uses the age-specific mean +2 SD to define “abnormal” values; however, this method will result in a similar proportion (∼5%) of extreme values across populations and lacks clinical meaning (9);

2. Prognostic significance, which assesses the future risk associated with the outcome;

3. Operational significance, which defines the level above which intervention will improve symptoms or prognosis (both of these approaches are clinically meaningful and require longitudinal data); and

4. Clinical significance, which denotes the level above which the frequency of symptoms or complications increases.

Fisher et al. (10) used the clinical significance approach to delineate cut points for the Diabetes Distress Scale (DDS) and the type 1 diabetes–specific DDS (T1-DDS) in adult populations (11,12). They examined linear and quadratic relationships between DD and clinical indicators of diabetes management (e.g., HbA_1c, physical activity) to identify changes in the gradient that might indicate a threshold for elevated DD. Increases of ∼0.5 SD demonstrated clinically meaningful differences in variables between DD categories (10). Three cut points were established for the T1-DDS, designating “little or no DD,” “mild,” “moderate,” and “high distress” (11). However, subsequent analysis using spline regression to demonstrate a change in the bivariate association with HbA_1c, revealed a single threshold for elevated DD (12).

Given there is now a validated measure of DD designed specifically for adolescents with type 1 diabetes—the PAID–Teens (PAID-T) (2)—but no established threshold for identifying elevated DD, we aimed to address this research gap by investigating clinically meaningful cut point(s) for the PAID-T scale that may identify adolescents experiencing elevated DD. Three variables were chosen to evaluate the clinical significance of elevated DD: depressive symptoms (Patient Health Questionnaire for Adolescents [PHQA]-8), HbA_1c, and frequency of self-monitoring of blood glucose (SMBG). These were selected because they are important emotional and diabetes-specific indicators and behaviors correlated with DD among adults (11,12) and adolescents with type 1 diabetes (3).

The Diabetes Management and Impact for Long-term Empowerment and Success (MILES) Youth–Australia Study (MILES Youth) was a national survey assessing the effect of diabetes on various psychosocial outcomes of Australian youth with type 1 diabetes and their parents. Full details of the survey methods were published previously (13). In brief, postal invitations to participate in a national, cross-sectional survey were sent to 5,928 registrants (59%) of the National Diabetes Services Scheme (NDSS) aged 10–19 years who had previously consented to be contacted for research purposes. The NDSS is an Australian Government initiative that provides subsidized products (e.g., insulin syringes, blood glucose test strips), information, and support services for Australians diagnosed with diabetes. All young people with type 1 diabetes are registered with the scheme and have access to subsidized multidisciplinary diabetes care through public hospital–based services or private practitioners. The study was also promoted through diabetes clinics, social media, and relevant diabetes magazines and newsletters. Commencing in August 2014, the survey was available online for 8 weeks. Before proceeding to the survey, participants were required to indicate their consent to participate. The MILES Youth study received ethical approval from the Deakin University Human Research Ethics committee (reference number 2014-060).

The current study includes MILES Youth participants aged 13 to 19 years with type 1 diabetes (n = 551) who had received the full survey, including the measures of DD and depressive symptoms. These measures were not presented to the younger group, aged 10–12 years (n = 230), owing to the nature of the survey method. The flow diagram of recruitment and inclusion/exclusion criteria is shown in Supplementary Fig. 1.

Participants reported their age, sex, country of birth, and postcode, which was used to index socioeconomic status (SES) (14). In addition, insulin delivery mode (pump or injections), diabetes duration, usual SMBG frequency (using an 8-point scale, “less than once a day” to “more than 7 times a day”) and most recent HbA_1c were self-reported. Respondents were asked to indicate when their last HbA_1c was checked (<3, 3–6, 7–12, >12 months, don’t know), and 69% indicated their HbA_1c was measured within the previous 3 months. The minimum recommended SMBG frequency is four times daily. At the time of the survey, continuous glucose monitoring was not subsidized for this age-group and not widely used and so was not included. Ten participants did not complete SMBG items and 80 did not report their HbA_1c. HbA_1c was excluded for participants with extreme values/outliers (n = 5).

Participants completed the PAID-T, a 26-item scale adapted from the adult version, which assesses the perceived emotional burden of living with diabetes (2). Items are rated on a 6-point scale: 1–2, not a problem; 3–4, a moderate problem; or 5–6, a serious problem. Item scores were summed to form a total score (range 26–156), with higher scores indicating greater DD (2). Because the adult version of the PAID is reported as a standardized score (0–100), we transformed the PAID-T using the formula (sum total – 26/130 × 100) so that the cut points were comparable. The final cut points are reported for the “PAID-T total” and “standardized PAID-T total” scores. We examined the factor structure using principal components analysis, which supported a unidimensional construct, consistent with the original study reporting the psychometric properties of the PAID-T (2). The scree plot suggested one principal component, which accounted for 49.5% of the total variance (eigenvalue: 12.9). The other components with eigenvalues >1.0 explained 6.9% and 5.5% of the variance respectively. All items had high loadings onto a forced single component (all coefficients >0.50). The Cronbach α was 0.96, indicating very high internal consistency reliability.

The PHQA-8 (15) was used to assess the presence and severity of depressive symptoms. The suicidal ideation item was omitted, in accordance with accepted procedures in population surveys (16) and the previously reported problematic nature of this item in the adult version (17). The eight items are scored from 0 (not at all) to 3 (nearly every day). Total scores range from 0 to 24, with higher scores indicating greater depressive symptoms. Cut points at 5, 10, 15, and 20 represent mild, moderate, moderately severe, and severe depressive symptoms respectively. The Cronbach α in this sample was 0.90.

Data were analyzed using SPSS 23 software for Windows (IBM Corp., Armonk, NY). Only complete data for the psychological measures were included. Data were checked for normality. Descriptive statistics are reported as mean ± SD or count and percentage. Associations between variables were examined using Pearson correlations and checked using nonparametric tests (Spearman rank) as appropriate for the data distribution.

After appraising various methods to establish a cut point, we adopted the clinical significance approach (9). We considered two methods for quantifying the level of DD: 1) the total number of items endorsed as “a serious problem” (i.e., rated 5–6) and 2) the summed PAID-T score. To determine whether the number of items endorsed would adequately capture the variation in DD levels, scatter plots were used to examine the PAID-T total score and the number of items endorsed in each of the three categories (not a problem, a moderate problem, a serious problem).

The pattern of the relationships between DD and the dependent variables (depressive symptoms, self-reported HbA_1c, and SMBG) were examined in separate multivariate regressions. Controlling for significant confounders (age, sex, and insulin pump use), we added linear and quadratic terms for DD in separate steps. Graphs of the quadratic regression fit lines were examined to identify one or more points of inflection that would indicate a potential cut point. The PHQA-8 summed score was logarithmically transformed before the regression analysis was performed to correct for a positively skewed distribution.

To identify points at which the PAID-T showed a marked increase or the trajectory changed, we plotted the mean PAID-T at each increment of the 1) PHQA-8 score (0–24), 2) HbA_1c levels (1.0% increments from 5.0 to 11.9%; and 12.0 to 15.9%), and 3) SMBG frequency. Differences in variables were examined by level of DD using one-way ANOVA. Levene tests revealed that the homogeneity of variance assumptions were not met; therefore, we used the Welch F test and the Games-Howell procedure for pairwise comparisons. Results were confirmed with nonparametric tests (Kruskal-Wallis and Jonckheere-Terpstra) but are not reported unless the results were different. Differences between means were considered statistically significant at P < 0.01 and clinically significant at >0.5 SD units or HbA_1c >0.5%. Effect sizes were calculated using the Cohen d and considered to be small (d = 0.2), medium (d = 0.5), or large (d = 0.8) (18).

The study included 537 adolescents with type 1 diabetes (Supplementary Fig. 1). Their mean age was 15.7 ± 1.9 years, and 334 (62%) were girls (Table 1). Mean diabetes duration was 6.5 ± 4.5 years, and 266 (50%) managed their diabetes with an insulin pump. The mean self-reported HbA_1c was 8.1% ± 1.5% (65 ± 16 mmol/mol). The mean PAID-T total score was 77 ± 30.

The PAID-T total score correlated positively with depressive symptoms (r = 0.62, P < 0.001) and HbA_1c (r = 0.34, P < 0.001) and negatively with SMBG frequency (r = −0.28, P < 0.001) (Supplementary Table 1). There was a small correlation between PAID-T and age (r = 0.17, P < 0.001) but no relationship with diabetes duration (P = 0.33). Girls reported higher DD than boys (83 ± 31 vs. 66 ± 27, P < 0.001). There was no difference in PAID-T by insulin delivery mode (P = 0.99) or SES (P = 0.41).

The scatter plots in Fig. 1 show that adolescents who rated most items 1–2 (“not a problem”) had a low PAID-T score (Fig. 1A), and those who rated most items 5–6 (“serious problem”) had a high PAID-T total score (Fig. 1C). However, items endorsed as a “moderate problem” (rated 3–4) contributed substantially to the total PAID-T score (Fig. 1B). Thus, quantifying the severity of DD using only items endorsed as a “serious problem” would ignore some adolescents experiencing substantial levels of distress (i.e., through most items being rated as a “moderate problem”). Further, item-level analysis (Pearson correlations) (Supplementary Table 1) showed that the PAID-T items most frequently endorsed as a “serious problem” were not the most strongly correlated with the psychological and clinical variables of interest, suggesting that this method is not the most appropriate for determining clinical significance. We therefore used the total score rather than the number of items endorsed for the cut point analysis.

There was a significant linear relationship between DD and log-transformed PHQA-8 (R² = 0.37, t = 14.18, P < 0.001), HbA_1c (R² = 0.12, t = 7.02, P < 0.001), and SMBG (R² = 0.16, t = −6.38, P < 0.001). When added to the model, the quadratic term for DD was not significant for any of the dependent variables; therefore, the trend line did not indicate a cut point.

The means plot in Fig. 2A shows an increase in the mean PAID-T in relation to PHQA-8 scores ≥70 and >90. PAID-T scores <70 corresponded with mean PHQA-8 ≤4 (Fig. 2A), indicating none-to-minimal depressive symptoms, HbA_1c <7.0% (Fig. 2B), and an average of ≥6 SMBG checks per day (Fig. 2C). PAID-T >90 corresponded with PHQA-8 scores >10 (moderate-to-severe depressive symptoms), high self-reported HbA_1c (≥9.0%; >75 mmol/mol), and less frequent SMBG than recommended (≤2 checks per day). These two cut points differentiated young people with optimal and suboptimal self-management and with or without depressive symptoms. As determined from these thresholds, 46% (n = 247) reported none-to-mild DD (PAID-T <70), 18% (n = 98) reported moderate DD (PAID-T 70–90), and 36% (n = 192) experienced high levels of DD (PAID-T >90) (Table 2). Cut points using standardized PAID-T scores were none-to-mild DD, <34; moderate DD, 34–50; and high DD, >50.

There were statistically significant differences between DD categories for mean PAID-T (F_2, 333.8 = 1,173.2, P < 0.001), PHQA-8 (F_2, 236.4 = 109.6, P < 0.001), HbA_1c (F_2, 212.4 = 21.7, P < 0.001), and SMBG frequency (F_2, 261.9 = 13.5, P < 0.001) (Table 2). Pairwise comparisons showed moderate-to-large effect sizes (Table 2). Differences between categories were >0.5 SD for depressive symptoms and HbA_1c >0.5% for those with high DD, indicating that the thresholds distinguished clinically relevant differences in these variables. SMBG frequency was significantly lower among those with high DD.

This study is the first to investigate the clinical significance of cut points to determine meaningful elevation of DD in adolescents (aged 13–19 years) with type 1 diabetes. These cut points show that 18% of participants reported moderate DD and 36% reported high DD. These thresholds suggest that DD is associated with psychological and clinical factors at a relatively low level in adolescents compared with adults (standardized PAID-T vs. standardized PAID: 34 vs. 40).

The items with the highest mean scores (i.e., most frequently endorsed as a “moderate” or “serious problem”) were not the most strongly correlated with psychological and clinical indicators, thus modal distribution and statistical approaches were not appropriate for determining a clinically meaningful cut point for elevated DD. Our analysis confirmed the original one-factor structure of the PAID-T (2). Further, the total score proved a better measure of elevated DD than items endorsed as a “serious problem,” which did not account for widespread “moderate” levels of distress that contributed substantially to the total score. A cut point was not found using the predicted mean, which increased linearly. However, we found a consistent trend in the relationships between the mean PAID-T total score and the three outcome variables, which enabled us to distinguish two cut points and three levels of severity.

Thresholds for elevated DD will be useful in a clinical setting for interpreting the PAID-T score and for initiating a conversation about DD. Young people with elevated DD had significant psychological impairment, which may affect their capacity to self-manage effectively. The adolescent age-group faces multiple biological, psychological, and social challenges, which can make managing diabetes especially difficult, and may need more support than is currently available to them. They may, for example, benefit from diabetes self-management education and support, which can enhance coping skills and self-efficacy (3), although such programs targeted at young people are not widely available in Australia. Psychoeducation programs and cognitive behavioral therapy have been shown to reduce DD among adolescents (3) and adults with type 1 and type 2 diabetes (19–21). Referral to a mental health professional is warranted for depression or anxiety. However, more than half of adolescents may have elevated DD; therefore, all diabetes clinicians have a role in recognizing and minimizing distress. Assessing DD by using age-appropriate tools, such as the PAID-T, could be integrated into the diabetes education and management plan. Individual PAID-T items reveal the sources of distress, which can facilitate the conversation and shared decision making about the most appropriate strategies for reducing DD and improving outcomes.

In a research context, a cut point provides criteria for inclusion into studies and the means to identify the magnitude of the problem (to target research and funding) and the effect of the intervention. It also enables comparison of prevalence across populations. Although the moderate group was modest in size, the data clearly indicated a middle category. Consistent with a study in adults (12), even moderate levels of DD were associated with depressive symptoms. Although younger age and DD are correlated in adults with type 1 diabetes (11,22), Lašaitė et al. (8) reported higher DD in emerging adults than in adolescents. Although differences may be attributable to the DD measures, methods of estimating thresholds, and the actual cut points, that there are real distinctions related to age and life stage is likely, supporting the need to use age-appropriate measures. DD appears to be highest among adolescents and young adults, who are challenged with managing diabetes during important life transitions (23).

Given the subjective nature of DD, there is no gold standard measure against which to validate the PAID-T. HbA_1c was used as a clinical outcome because it is a standardized indicator of diabetes management and complication risk. A study using clinic-recorded HbA_1c reported a threshold for increased mood disorders among children (8.7%) that was consistent with our findings (24). However, broader aspects of living with diabetes (e.g., social impact) may be unrelated to HbA_1c. Although DD has a moderately strong correlation with the PHQA-8 scale, the latter is a measure of general emotional distress and unlikely to capture the full experience of living with diabetes. Further, a small proportion of people with type 1 diabetes who experience elevated depressive symptoms do not report high levels of DD (25). Thus, HbA_1c and depressive symptoms both have limited validity as a gold standard in this context, but determining cut points with reference to both biomedical and general psychological indicators minimizes any bias that might be introduced with a single reference. Further research is needed to investigate the most appropriate indicators to explore DD cut points.

The strengths of this study include the large national sample, the use of an adolescent-specific measure of DD, and the data-driven approach to defining thresholds for elevated DD. Several methodological limitations need to be considered and have been discussed in greater detail elsewhere (13). The MILES Youth study participants were self-selected, and the overall response rate was low (13%). Respondents were not representative of the Australian adolescent population with type 1 diabetes because they were relatively advantaged socioeconomically and boys were somewhat underrepresented (13). We found that DD was unrelated to SES, but girls reported higher DD than boys, and this may have influenced our results. The mean PAID-T score for our sample (77 ± 30) was consistent with the score in a recent U.S. study (73 ± 27) in a very similar age-group (range 14–18 years) (26). Furthermore, similar sex differences in DD have been identified among adolescents in the U.S. (2,26). HbA_1c and SMBG were self-reported and thus subject to recall and social-desirability bias, and a large number of respondents did not provide an HbA_1c. Nonetheless, compared with meter download, self-reported SMBG has shown satisfactory reliability (27) and fewer-than-expected errors (28). The mean self-reported HbA_1c among this sample was only slightly lower than that found in a clinic-based study in a recent study in Victoria (8.1% vs. 8.3%) (29), and the strength of the association between DD and HbA_1c was similar to that observed in other studies among adolescents (3), despite not being collected concurrently with the PAID-T (69% of HbA_1c checks had been conducted within the past 3 months). Any self-report and sample bias is minimized because the cut point analysis focused on those at greatest risk (SMBG <2 checks per day; HbA_1c >9%).

In summary, this study has established cut points for moderate and high DD, which will be useful for clinical and research purposes. Most of the adolescents experienced at least moderate DD, which was shown to be clinically significant. These cut points provide the opportunity, for the first time, to identify clinically significant levels of DD among adolescents with type 1 diabetes to enable interventions to be targeted for those most in need of support. Confirmation of these cut points in other populations and with other indicators is warranted. Exploration of DD in longitudinal studies would be useful to assess the prognostic value of the cut points and their stability over time.

Acknowledgments. The authors thank the adolescents with type 1 diabetes who participated in this study. The authors thank Dr. Jill Weissberg-Benchell (Northwestern University, Chicago) for providing permission for use of the PAID-T in this study and acknowledge the assistance of Dr. Sharmala Thuraisingam (University of Melbourne) for advice on statistical analysis.

Funding. The Diabetes MILES Youth–Australia 2014 Survey was funded by the NDSS, an initiative of the Australian Government administered with assistance from Diabetes Australia. C.H. and J.S. are supported by core funding to The Australian Centre for Behavioural Research in Diabetes derived from the collaboration between Diabetes Victoria and Deakin University.

Duality of Interest. No potential conflicts of interest relevant to this article were reported.

Author Contributions. V.H. analyzed the data and wrote the initial draft of the manuscript. V.H., C.H., F.C., F.P., T.C.S., and J.S. commented on the initial manuscript draft, contributed to subsequent revisions, and approved the final manuscript. V.H., C.H., and J.S. planned the analysis. C.H. and J.S. reviewed and edited the manuscript. V.H. and J.S. are the guarantors of this work and, as such, had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.