PedsQL 3.2 Diabetes Module for Children, Adolescents, and Young Adults: Reliability and Validity in Type 1 Diabetes

The international incidence and prevalence of type 1 diabetes have increased significantly over the past several decades in children, adolescents, and young adults (1–3). Measuring the patient’s perspective of the impact of diabetes on daily living has become even more important as the emphasis on personalized medicine and tailored interventions requires shared decision making between the patient and health care providers regarding risk assessment and evidence-based treatment choices (4).

The Pediatric Quality of Life Inventory (PedsQL) 3.0 Diabetes Module (www.pedsql.org) is one of the most widely used internationally validated patient-reported outcome (PRO) measures to assess diabetes-specific health-related quality of life (HRQOL) of children, adolescents, and young adults from the perspective of both pediatric patients and their parents (5–8). Prior studies using the PedsQL 3.0 Diabetes Module have demonstrated the measure’s reliability and validity in children, adolescents, and young adults with type 1 diabetes (6,8). Nonetheless, the PedsQL 3.0 Diabetes Module was developed almost 20 years ago, and evolving differences in treatment regimens and the use of the measure in young adult patients warranted additional qualitative methods research on the content validity of the instrument for current use.

To establish contemporary content validity, we used qualitative methods as recommended by the U.S. Food and Drug Administration (9) and the PRO measurement literature (10,11) to develop new items and revise and delete existing items as needed, resulting in the PedsQL 3.2 Diabetes Module (12). However, we have not previously reported on the measurement properties of the new 33-item PedsQL 3.2 Diabetes Module using quantitative methods. Further, previous investigators using the PedsQL 3.0 Diabetes Module have recommended a total score comprising all 28 items as most parsimonious based on factor analysis rather than the a priori five-scale structure, given the lower internal consistency reliability of some of the a priori scales (13). Nevertheless, a total score combines both diabetes symptoms and diabetes management items into one unitary summary score, which may not be the most accurate and precise representation of the constructs manifested by the items.

Consequently, the objective of the current study was to describe the feasibility, internal consistency reliability, construct validity, and minimal clinically important difference (MCID) scores of the 33-item PedsQL 3.2 Diabetes Module for children, adolescents, and young adults with type 1 diabetes from a 10-site national field test study, designating a hypothesized two-factor structure comprising a Diabetes Symptoms Summary Score and Diabetes Management Summary Score as two new summary scores representing the constructs measured by the items.

Patients aged 2–25 years with type 1 diabetes were recruited from 10 clinical sites in the U.S. (see Supplementary Data). A total of 656 families participated (Table 1). Data collection for the field test took place between July 2015 and June 2017. Parental informed consent and patient assent/consent (when age appropriate) were obtained. The research protocol was approved by the institutional review board at each site.

The 33-item PedsQL 3.2 Diabetes Module is a diabetes-specific HRQOL instrument consisting of five a priori scales measuring Diabetes Symptoms (15 items), Treatment Barriers (5 items), Treatment Adherence (6 items), Worry (3 items), and Communication (4 items). Cognitive interviewing techniques were used to add, delete, and/or revise the items of the existing PedsQL 3.0 Diabetes Module as needed based on individual interviews with children, adolescents, and young adults with type 1 diabetes (12). In addition to rewording some items for greater clarity based on the cognitive interviews, 7 new items were added and 2 items deleted from the 28-item PedsQL 3.0 Diabetes Module resulting in 33 items, with the recall period changed from 1 month to 7 days based on patient feedback (12). Specifically, we added the following four new diabetes symptoms items: “I feel like I need to throw up,” “I go high,” “I feel dizzy,” and “I feel weak.” We reworded “I feel tired or fatigue” to “I feel tired” as “fatigue” was not well understood by all patients. We reworded “I feel irritable” to “I feel cranky or grumpy” because “cranky or grumpy” was better understood by most patients. The only difference between the children, teen, and the young adult versions for the 15 diabetes symptoms was the item, “I have stomachaches.” For children, the item was worded as, “I have tummy aches.” For diabetes management, the items “I am embarrassed by my diabetes treatment” and “I worry about going high” were added and “I worry about whether or not my medical treatments are working” was deleted based on patient input. “It is hard for me to wear my id bracelet” was deleted as it was no longer deemed relevant. A double-barreled question (measuring two constructs), “It hurts to prick my finger or give insulin shots,” was reworded into two separate single construct items, “It is hard for me to take glucose tests” and “It is hard for me to take insulin shots,” resulting in an additional item. The item “It is hard for me to stick to my diabetes care plan” was reworded to “It is hard for me to do everything I need to do to care for my diabetes” to improve clarity based on patient feedback as “diabetes care plan” was not terminology used by most patients. The item “It is hard for me to exercise” was reworded for child self-report to “It is hard for me to play or do sports” and for adolescent self-report to “It is hard for me to exercise or do sports.” For young adults, this item reads, “It is hard for me to exercise.” These changes were made to be more relevant to children and adolescents based on their input. For the item, “It is hard for me to keep track of carbohydrates or exchanges,” the words “or exchanges” were deleted as “exchanges” was not well understood. For the item, “It is hard for me to eat snacks,” the phrase “when I go low” was added as patients expressed the need for this clarity. The item now reads, “It is hard for me to snack when I go low.” For ages 2–7 years, the item regarding worrying about long-term complications from diabetes is not included for both patient self-report and parent proxy report as it is a concept too difficult for the younger age-groups to understand. Diabetes symptoms and management concerns elicited during the cognitive interviews were remarkably consistent across the age-groups tested (12). A concept tracking matrix showing the changes between revisions, age-appropriate wording differences, and the final items of the new 33-item PedsQL 3.2 Diabetes Module have been published in a content validity study (12). The items for teen self-report are listed in Table 2.

The PedsQL 3.2 Diabetes Module is composed of parallel patient self-report and parent proxy report formats for ages 5–25 years and a parent proxy report format for ages 2–4 years. Patient self-report forms are specific for ages 5–7, 8–12, 13–18, and 18–25 years. Parent proxy report forms are specific for ages 2–4 (toddler), 5–7 (young child), 8–12 (child), 13–18 (adolescent), and 18–25 (young adult) years and assess patient’s and parent’s perceptions of the patient’s diabetes-specific symptoms and management problems. One adult from each family (70.3% mothers, 14.3% fathers, 2.9% grandparents, 1.1% guardians, 3.8% other, 7.6% missing) completed the proxy report version for the current study. The items for each of the forms are essentially identical, differing in developmentally appropriate language, or first or third person tense. The instructions ask how much of a problem each item has been during the past 7 days. The grammar and syntax of the new items are structurally equivalent to those in the existing PedsQL item bank (12). The 5-point Likert-type response scale is the same as the existing PedsQL 3.0 Diabetes Module, with items reverse-scored and linearly transformed to a 0–100 scale (0 = 100, 1 = 75, 2 = 50, 3 = 25, 4 = 0), so that lower scores demonstrate more diabetes symptoms and management problems and, hence, lower diabetes-specific HRQOL. Summary scores are computed as the sum of the items divided by the number of items answered (this accounts for missing data). If more than 50% of the items in the scale are missing, the summary score is not computed (14). This accounts for the differences in sample sizes for Diabetes Symptoms and Diabetes Management Summary Scores reported in the tables. Although there are other strategies for imputing missing values, this computation is consistent with previous PedsQL peer-reviewed publications, as well as other well-established HRQOL measures (15).

The 23-item PedsQL 4.0 Generic Core Scales encompass Physical Functioning (8 items), Emotional Functioning (5 items), Social Functioning (5 items), and School Functioning (5 items) (16). To create the Total Scale Score, the mean is computed as the sum of the items divided by the number of items answered in the Physical, Emotional, Social, and School Functioning Scales. The Total Scale Score measures overall generic HRQOL (16). Higher scores indicate better HRQOL.

Caregivers or older patients completed the PedsQL Family Information Form, which contains demographic information including the child’s date of birth, sex, and race/ethnicity (16).

HbA_1c values were measured at each data collection site using standard point-of-care methods (e.g., DCA Vantage Analyzer) and not at a central laboratory. The most recent HbA_1c value from the patient’s medical record was used.

BMI values were calculated from height and weight measures at each data collection site. BMI percentile values were calculated using the Centers for Disease Control and Prevention scoring algorithms (17).

A principal component factor analysis with promax rotation of the items was initially conducted with all 33 items to evaluate the factor structure of the PedsQL 3.2 Diabetes Module. We hypothesized that the diabetes symptoms and diabetes management items would represent two different factors (constructs) configured as two new summary scores representing the Diabetes Symptoms Summary Score and the Diabetes Management Summary Score. Based on recommendations from the measurement literature, we included items with a factor loading of 0.30 or greater (18).

Feasibility was determined from the percentage of missing values (19). Cronbach coefficient α was used to determine internal consistency reliability (20). Internal consistency reliabilities of 0.70 or greater are recommended for comparing patient groups, whereas an internal consistency reliability criterion of 0.90 is recommended for analyzing individual patient scores (21). Range of measurement was based on the percentage of scores at the extremes of the scaling range, that is, the maximum possible score (ceiling effect = percentage of scale scores at 100 [never a problem]) and the minimum possible score (floor effect = percentage of scale scores at 0 [almost always a problem]). Surveys with small floor or ceiling effects (1%–15%) are considered to meet acceptable measurement standards, whereas surveys with moderate floor or ceiling effects (>15%) are considered less precise in measuring constructs at the extremes of the scale (22).

The MCID was calculated using the SEM derived by multiplying the SD by the square root of 1 − α (Cronbach α reliability coefficient) (23). This equation, SEM = SD√ [1 – α], is a distribution-based methodology for determining the MCID and has been previously used to determine the MCID for the PedsQL 3.0 Diabetes Module (8), as well as other PedsQL modules (24). The SEM has been linked to the MCID, in which 1 SEM has demonstrated a strong correspondence to anchor-based individual change thresholds (25). The MCID is considered the smallest clinically meaningful change in a PRO score that can be detected with measurement precision for the construct (latent variable) and not as a result of measurement error (8,24,26). The MCID has been defined as “the smallest difference in score in the outcome of interest that informed patients or informed proxies perceive as important, either beneficial or harmful, and that would lead the patient or clinician to consider a change in the management” (27). Thus, the MCID provides evidence in support of the clinical interpretability of scale scores (24) and represents a change in scores that would be perceived as a clinically meaningful difference to patients and their parents (8).

The intercorrelations among the PedsQL 3.2 Diabetes Module Diabetes Symptoms Summary Score and the Diabetes Management Summary Score and PedsQL Generic Core Scales were used to examine construct validity (28). Based on the conceptualization of disease-specific symptoms as causal indicators of generic HRQOL (29), we hypothesized that greater diabetes symptoms and diabetes management problems would correlate with lower overall generic HRQOL. Pearson product-moment correlation coefficients effect sizes are designated as small (0.10), medium (0.30), and large (0.50) in magnitude (30). We also examined the associations between the Diabetes Symptoms Summary Score and the Diabetes Management Summary Score and age, sex, HbA_1c, BMI, BMI percentile, and time since diagnosis.

Intraclass correlation coefficients (ICCs) were used to determine agreement between patient self-report and parent proxy report (31). The ICC provides an index of absolute agreement as it takes into account the ratio between subject variability and total variability (32). ICCs are designated as ≤0.40 (poor to fair agreement), 0.41–0.60 (moderate agreement), 0.61–0.80 (good agreement), and 0.81–1.00 (excellent agreement). Statistical analyses were conducted using IBM SPSS (Armonk, NY).

Table 1 contains the demographic and clinical characteristics of the participants. Supplementary Table 2 contains the number of participants for each age-group (n = 4, ages 2–4 years; n = 21, ages 5–7 years; n = 200, ages 8–12 years; n = 375, ages 13–18 years; n = 49, ages 19–25 years; missing age variable = 7).

Based on the item factor loadings from the principal components factor analysis, two summary scores emerged that differentiated the items into the latent constructs “diabetes symptoms” and “diabetes management.” Specifically, as shown in columns 1–7 in Table 2, when all items were subjected to factor analysis, the a priori diabetes symptoms scale items loaded into subgrouping of items that were generally consistent with items measuring diabetes symptoms (demonstrating face validity), and the a priori scales measuring problems with diabetes management loaded into subgroupings of items that were generally consistent with items measuring diabetes management problems (demonstrating face validity).

To further support these two summary scores, we then conducted a principal components factor analysis designating a priori a two-factor structure. As shown in Table 2 in the columns labeled “Diabetes symptoms” and “Diabetes management,” the resulting factor loadings indicated that the Diabetes Symptoms Summary Score and the Diabetes Management Summary Score represented two different constructs. Even though a further exploratory factor analysis of the Diabetes Symptoms Summary Score suggested that the items might be additionally grouped into subscales or “facets” (33) measuring hypoglycemia and hyperglycemia, we determined that the cross-loadings of some items did not warrant demarcating the items further. Factor analysis of the items in the Diabetes Management Summary Score suggested that the Worry Scale items loaded as the a priori Worry Scale, whereas the other items loaded in subscales or facets not consistent with the a priori scales. Accordingly, we determined that the Diabetes Management Summary Score was the most parsimonious scoring method for the diabetes management items.

It should be noted that the PedsQL 3.2 Diabetes Module was primarily designed to measure patient self-reported HRQOL. Nonetheless, parent proxy report is considered a necessary complement for patient self-report given that parents’ perspectives may drive patient health care use, particularly for younger patients and those patients who cannot self-report. In this context, we also examined the parent proxy report factor loadings for the 33 items by conducting a principal components factor analysis designating a priori a two-factor solution. As shown in Supplementary Table 1, the two-factor solution for parent proxy report items is generally consistent with the two-factor solution for patient self-report.

Cronbach α internal consistency reliability coefficients for the Diabetes Symptoms and Diabetes Management Summary Scores are shown in Table 3. All patient self-report and parent proxy report summary scores approach, meet, or exceed the reliability criterion of 0.90 recommended for analyzing individual patient scores when combined across age-groups. As shown in Supplementary Table 2, when analyzing the individual age-groups, patient self-report for ages 5–7 years did not meet the 0.70 criterion for group comparisons, whereas patient self-report for ages 8–25 years meet, approach, or exceed the 0.90 criterion.

The percentage of missing item responses were 0.01% and 0.05% for patient self-report and parent proxy report, respectively.

Table 3 contains the percentage of scores at the extremes of the scaling range (floor and ceiling effects) for the Diabetes Symptoms and Diabetes Management Summary Scores. For patient self-report and parent proxy report, there were no significant floor effects (lower scores demonstrate more diabetes symptoms and diabetes management problems and hence lower diabetes-specific HRQOL) or ceiling effects (higher scores demonstrate less diabetes symptoms and management problems and hence higher diabetes-specific HRQOL).

Table 4 shows the correlations between the Diabetes Symptoms and Diabetes Management Summary Scores with the PedsQL 4.0 Generic Core Scales and Summary Scores. The majority of the correlations demonstrate large effect sizes (≥0.50) in magnitude (all P < 0.001), supporting construct validity.

We also examined the correlations between the Diabetes Symptoms and Diabetes Management Summary Scores. As anticipated, the Diabetes Symptoms and Diabetes Management Summary Scores were highly correlated (patient self-report r = 0.63, P < 0.001; parent proxy report r = 0.62, P < 0.001), similar to the correlations with the Generic Core Scales Total Scale Score. Even though the Diabetes Symptoms and Diabetes Management Summary Scores are significantly correlated, the data suggest that they are measuring distinct constructs as the percent variance accounted for (r²) are relatively modest (patient self-report r² = 0.40 [40% of the variance]; parent proxy report r² = 0.38 [38% of the variance]). Thus, although it would be expected that diabetes symptoms would be associated with diabetes management, the percent variance accounted for in diabetes symptoms by diabetes management indicates that other factors are relevant in explaining the remaining variance in diabetes symptoms. Also of note, Diabetes Symptoms Summary Scores were significantly lower (worse) than Diabetes Management Summary Scores (patient self-report 65.53 vs. 79.19, respectively, t[646] = −25.67, P < 0.001; parent proxy report 68.14 vs. 77.42, respectively, t[601] = −16.18, P < 0.001), further supporting two summary scores rather than one unitary total score for all the items.

Table 3 shows the MCID scores. These MCID values provide information on the clinical interpretability of the summary scores. For example, from Table 3, a patient self-reported Diabetes Symptoms Summary Score that changed greater than or equal to 5.55 is a numerical value indicating the smallest clinically meaningful change that can be detected. The other MCID values in Table 3 can be similarly interpreted.

Supplementary Table 3 contains the associations with clinical and demographic variables. HbA_1c was significantly correlated with the Diabetes Symptoms and Diabetes Management Summary Scores for both patient self-report and parent proxy report. BMI was not significantly correlated with the summary scores. BMI percentile was only significantly correlated with parent proxy reported Diabetes Symptoms and Diabetes Management Summary Scores. Time since diabetes diagnosis was not significantly associated with the summary scores. Age was not significantly correlated with the summary scores. Females demonstrated significantly lower (worse) summary scores than males for patient self-report and parent proxy report. Mothers proxy reported significantly worse Diabetes Symptoms Summary Scores for their children than fathers (68.03 vs. 73.60, respectively, t[545] = −3.11, P < 0.01), but no proxy-reported significant differences between mothers and fathers for Diabetes Management Summary Scores (77.86 vs. 78.82, respectively, t[546] = −0.55, P > 0.05) were demonstrated.

The ICCs between patient self-report and parent proxy report were 0.51 and 0.47 for the Diabetes Symptoms and Diabetes Management Summary Scores, respectively (P < 0.001). These ICC values represent moderate agreement.

These analyses support the feasibility, internal consistency reliability, construct validity, and MCID of the PedsQL 3.2 Diabetes Module Diabetes Symptoms and Diabetes Management Summary Scores. The PedsQL 3.2 Diabetes Module represents a revised and updated version of the widely used PedsQL 3.0 Diabetes Module and makes a significant contribution to the empirical literature by providing a contemporary measure of diabetes symptoms and diabetes management problems that are relevant to a broad age range.

It is essential to reemphasize that the previously recommended total score obscures the important conceptual distinctions between diabetes symptoms and diabetes management. In the current study, patients self-reported and parents proxy reported significantly lower (worse) Diabetes Symptoms Summary Scores than Diabetes Management Summary Scores. When reporting an averaged unitary total score, the differences between the diabetes symptoms versus diabetes management constructs are obfuscated as conceptually distinct PROs. Future studies should report the 15-item Diabetes Symptoms Summary Score and 18-item Diabetes Management Summary Score as separate outcomes in clinical trials and predictive analytics models rather than as one total score.

The MCIDs represent numeric values that indicate the magnitude of change in summary scores that are detectable by the patient and parent as a clinically meaningful difference in the constructs being measured and provide an important reference point that can be used in clinical research and practice (8,24). In the current study, the MCIDs for patient self-report and parent proxy report were 5.55 and 5.15 for the Diabetes Symptoms Summary Score and 5.07 and 5.05 for the Diabetes Management Summary Score, respectively. These MCIDs are comparable to the 5.27 for patient self-report and 4.54 for parent proxy report for the unitary total scale score as reported by Hilliard et al. (8) for the PedsQL 3.0 Diabetes Module in type 1 diabetes. Notably, the MCIDs for the 11-item version 3.0 Diabetes Symptoms Scale for patient self-report (7.56) and parent proxy report (5.81) from Hilliard et al. were somewhat larger and hence less precise than the current 15-item 3.2 version for the Diabetes Symptoms Summary Score. The MCIDs for the Diabetes Management Summary Scores for patient self-report and parent proxy report from the current study are considerably smaller and hence more precise than the individual 3.0 versions of the Treatment Barriers, Treatment Adherence, Worry, and Communication Scales reported in Hilliard et al. (8) for patient self-report (10.86, 9.99, 12.01, 7.53, respectively) and parent proxy report (10.57, 9.56, 8.52, 7.91, respectively), suggesting that the Diabetes Management Summary Score is a more precise MCID.

The current study has several strengths, including the rigorous methods used to construct the instrument, the relatively large overall sample size, the broad age range of participants, and the nationwide representation of the participants. Rates of insulin pump and continuous glucose monitor use in the sample are similar to published data from the T1D Exchange Clinic Registry (34), further supporting the representativeness of the sample. Limitations include the lack of information on families who chose not to participate, the lack of information on the percentage of participants who adhered to and met American Diabetes Association clinical guidelines, the absence of test/retest reliability, an unidentified standard window of time around the date that the youth completed the PedsQL 3.2 Diabetes Module and the most recent HbA_1c value was obtained, and the lack of information on the implementation/administration feasibility across clinic sites including recruitment rates, length of time for completion, and completion by paper/pencil or electronic version of the questionnaires. Additionally, construct validity was assessed through intercorrelations with generic HRQOL. Although an acceptable approach to validating a disease-specific HRQOL measurement instrument, future research should also include other standardized measures of psychosocial functioning. The sample size for ages 2–4 and ages 5–7 years were small, rendering the analyses of these two age-groups as preliminary. Additionally, patient self-report for ages 5–7 years did not attain conventional levels of internal consistency reliability, and additional research with larger sample sizes are needed for this age-group. Until that time, parent proxy report should be used for ages 5–7 years. An additional limitation was the lack of a central laboratory for HbA_1c measurement across the 10 sites, although the majority of the sites used a similar measurement approach. In testing conceptual or “mechanistic” models, one potential limitation of the Diabetes Management Summary Score is the inclusion of several constructs that may overlap with other measures in these models. For example, in testing conceptual models of treatment adherence using a specifically designed scale that measures diabetes adherence, it would be recommended to either separate out the items that measure this construct from the Diabetes Management Summary Score or simply to use only the Diabetes Symptoms Summary Score in the conceptual model. Finally, the items included in the PedsQL 3.2 Diabetes Module were developed through qualitative methods as recommended by the U.S. Food and Drug Administration, which included the patients’ and their parents’ perspective on the content of the items (12). This approach did not result in content covering newer diabetes technologies, as reflected in the lack of items on newer diabetes technologies in the Diabetes Management Summary Score items. When working with pediatric patients and families, it may be important to include other measures specifically focused on diabetes devices and technologies when administering the PedsQL.

Future studies using the Diabetes Symptoms and Diabetes Management Summary Scores should facilitate a more precise understanding of diabetes-specific HRQOL. When evaluating new and existing therapeutic interventions, the PedsQL 3.2 Diabetes Module Diabetes Symptoms and Diabetes Management Summary Scores may help clinicians and researchers identify individuals and patient groups with different diabetes-specific HRQOL profiles that may be used when individualizing patient-centered care. In sum, the Diabetes Symptoms and Diabetes Management Summary Scores represent standardized PROs of diabetes symptoms and diabetes management that may be useful for clinical research, clinical trials, and practice in children, adolescents, and young adults.

Acknowledgments. The investigators thank the following individuals for their involvement in participant recruitment, data collection, and/or data verification: Vincent Aguirre, Marta Pardo, and Morgan Drake (Texas A&M University); Natalie Beauregard and Cisco Pascual (Seattle Children’s Research Institute); Nora Chokr and Marie Nader (University of California, San Francisco); Jacqueline Shea and Kylie Benson (University of Colorado Denver); Lisa Keys and Elizabeth Dabrowski (Northwestern University Feinberg School of Medicine); Esti Iturralde and Bianca Agustin (Stanford University School of Medicine).

Funding and Duality of Interest. No funding was specifically designated for this field test study data collection effort or manuscript preparation. New item development and item modification of the existing PedsQL 3.0 Diabetes Module for the published item generation qualitative methods study for the PedsQL 3.2 Diabetes Module was previously funded by Eli Lilly and Co., Indianapolis, IN. J.W.V. holds the copyright and the trademark for the PedsQL and receives financial compensation from the Mapi Research Trust, which is a nonprofit research institute that charges distribution fees to for-profit companies that use the PedsQL. He did not receive funding from Eli Lilly and Co. for the current quantitative methods field test study. No other potential conflicts of interest relevant to this article were reported.

Author Contributions. J.W.V. was involved in the conception and design of the study, the writing of the first draft of the manuscript, the statistical analysis of the data, and the interpretation of the data. A.M.D., K.K.H., J.K.R., N.T.C., K.A.D., J.C.W., J.P.Y.-F., E.K.G., M.A.F., S.D.C., J.C.K., J.L.M., and D.P.W. were involved in the conception and design of the study, the acquisition of the data, the interpretation of the data, and the critical revision of the manuscript for important intellectual content. E.M.D. and R.W.H. were involved in the conception and design of the study, the interpretation of the data, and the critical revision of the manuscript for important intellectual content. All authors approved the final version of the manuscript. J.W.V. is the guarantor of this work and, as such, had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.