OBJECTIVE—It is recommended that the psychological status of adolescents with diabetes be assessed periodically as part of ongoing care. The World Health Organization-Five Well-Being Index (WHO-5) is a short self-report instrument that appears suitable for this purpose. This study is the first to assess the reliability and validity of the WHO-5 in adolescents with type 1 diabetes.
RESEARCH DESIGN AND METHODS—Ninety-one adolescents with type 1 diabetes (aged 13–17 years) from four pediatric clinics completed the WHO-5, along with other psychological measures: the Center for Epidemiologic Studies Depression Scale (CES-D), the Diabetes Family Conflict Scale (DFCS), and the mental health and self-esteem subscales of the Child Health Questionnaire (CHQ-CF87). Confirmatory factor analysis (CFA) and exploratory factor analysis were conducted. Readability, homogeneity, and item-total and inter-item correlations were determined. Concurrent validity was examined by calculating correlation coefficients among all measures. Sensitivity and specificity of the WHO-5 were tested against those for the CES-D using receiver operating characteristic (ROC) curves.
RESULTS—CFA confirmed the one-factor structure; Cronbach's α of this 5-item scale was 0.82. The WHO-5 showed a moderate to strong correlation with the CES-D (r = −0.67), with the mental health (r = 0.60) and self-esteem (r = 0.43) subscales of the CHQ-CF87, and with the DFCS (r = −0.34), confirming concurrent validity. ROC curve analysis confirmed the WHO-5 cutoff point of <50 for identification of mild to severe depressive affect (sensitivity 89% and specificity 86%).
CONCLUSIONS—The WHO-5 is a brief, patient-friendly measure of positive well-being with good psychometric properties that appears suitable for routine use in adolescents with type 1 diabetes.
Depression appears to be two to three times more prevalent in adolescents with diabetes compared with adolescents in the general population and adversely affects quality of life and diabetes outcomes (1–4). It is therefore recommended that screening for depression be performed routinely in this age-group, but there is no consensus on which measure to use for this purpose (1–3,5). The World Health Organization-Five Well-Being Index (WHO-5) is a short, positively worded instrument designed to assess the level of emotional well-being over a 14-day period. The screening properties of the WHO-5 have been studied in adults with and without diabetes against the Structured Clinical Interview for the Diagnostic and Statistical Manual (DSM)-IV, showing excellent sensitivity (94–100%) and specificity (78%) (6,7). Given its brevity and focus on positive affect, the WHO-5 may be a suitable instrument to screen for low emotional well-being and depressive affect in adolescents with diabetes. Moreover, as the WHO-5 is a generic measure, it allows for comparison with healthy peers and is available in a multitude of languages. However, only one study so far used the WHO-5 in young men, but no validation data were reported (8). We therefore set out this study to investigate the reliability and validity of the WHO-5 in adolescents with type 1 diabetes.
RESEARCH DESIGN AND METHODS—
Participants between the ages of 13 and 17 years were recruited from four pediatric outpatient clinics in the Netherlands as part of a larger psychosocial research project. Insufficient language skills and diabetes duration <6 months were exclusion criteria. Of the total 171 eligible subjects, 91 adolescents with type 1 diabetes consented to participate. The study was approved by the medical ethical committees of all hospitals, and written informed consent was obtained from both patients and parents. The adolescents who chose not to participate did not differ in age, sex, or A1C from participating adolescents. Nonparticipants were, however, more likely to be from an ethnic minority.
The WHO-5 captures emotional well-being and was developed from the World Health Organization-Ten Well-Being Index (9,10). It was conceptualized as a unidimensional measure that contains five positively worded items: “I have felt cheerful and in good spirits;” “I have felt calm and relaxed;” “I have felt active and vigorous;” “I woke up feeling fresh and rested;” and “My daily life has been filled with things that interest me.”
The degree to which the aforementioned positive feelings were present in the last 2 weeks is scored on a 6-point Likert scale ranging from 0 (not present) to 5 (constantly present). The raw scores are transformed to a score from 0 (worst thinkable well-being) to 100 (best thinkable well-being). A score <50 suggests poor emotional well-being and is a sign for further testing. A score ≤28 is indicative of depression (6). In adults, the WHO-5 proved to be a highly sensitive screener for depressive affect (6,7).
In addition to the WHO-5, the Center for Epidemiologic Studies Depression Scale (CES-D), two subscales of the 87-item child report version of the Child Health Questionnaire (CHQ-CF87) evaluating mental health and self-esteem, and the Diabetes Family Conflict Scale (DFCS) were completed.
The CES-D consists of 20 items (4 positive and 16 negative) and is scored from 0 (never) to 3 (daily) on the basis of frequency of depressive symptoms reported in the past week (11). Total CES-D summation scores range from 0 (no depressive symptoms) to 60 (most frequent/severe depressive symptoms). In adults, a cutoff score of 16 is used to define likely depression. However, this criterion has yielded mixed results in adolescents (12–15), and a cutoff of 24 was suggested to improve the correlation with depression defined according to the DSM-III revised (16). Similar to the U.S. National Longitudinal Study of Adolescent Health and the SEARCH for Diabetes in Youth Study, we adapted this cutoff score and stratified depressive affect as “minimal” (0–15), “mild” (16–23), and “moderate/severe” (≥24) (3,17).
The mental health (16 items) and self-esteem (14 items) subscales of the CHQ-CF87 are scored from 1 to 5 and transformed to a score between 0 and 100, with higher scores representing better well-being. The ratings of all scales are based on children's functioning over the previous 4 weeks (18).
The version of the Diabetes Family Conflict Scale (DFCS) adapted by Laffel and colleagues assesses the current degree of family conflict on 19 management tasks rated on a 3-point Likert scale (19). The DFCS does not specify a time frame. Higher scores indicate more conflicts.
Because the WHO-5 was originally developed for adults, it is important to assess the readability of the measure. For this purpose, we calculated the Flesch Reading Ease score. This score is based on the average number of syllables per word and the average sentence length. Higher scores represent better readability; eighth grade students could easily understand passages with a score of 70 (20).
Confirmatory factor analysis (CFA) was conducted by structural equation modeling to confirm the factor structure established in adult studies, using Mplus (version 3.13). The χ2, root mean square error of approximation (RMSEA), and comparative fit index (CFI) were calculated. RMSEA values between 0.08 and 0.10 and CFI values >0.90 are indicative of adequate fit. An exploratory factor analysis using principal component analysis was used to study the appropriateness of this factor structure in our adolescent population. Only components with eigenvalues >1.0 should be retained, together explaining >50% of the total variance and showing factor loadings >0.40 (21). Internal consistency by Cronbach's α and item-total and interitem correlations were calculated to assess the reliability of the scale. For internal consistency, an α of 0.70–0.80 is desirable, and the item-total correlation should be >0.20. A high interitem correlation (>0.80) is an indication of redundancy and is therefore not desirable. In contrast, if all correlations are near 0, there is no meaningful construct (22).
Concurrent validity was examined by calculating Pearson and Spearman correlation coefficients among the questionnaires. For similar constructs, correlations >0.50 are an indication for good concurrent validity. These correlations could be expected for the CES-D and mental health subscale of the CHQ-CF87. Poor well-being is expected to be associated with more family conflicts and low self-esteem, although not to the extent that it measures the same construct. We therefore expect moderate correlations between 0.30 and 0.50 (23). The sensitivity (proportion of truly diseased individuals identified) and specificity (proportion of truly nondiseased individuals identified) were investigated using receiver operating characteristic (ROC) curve analyses. The cutoff scores used in adult populations (28 and 50) were related to the cutoff points of the CES-D (respectively, 24 and 16).
In addition to the reliability and validity, differences in sex, ethnicity (Caucasian or not), one- or two-parent family, and treatment regimen were tested using ANOVA. Correlations between the WHO-5, age, A1C, BMI, and diabetes duration were explored using Pearson correlation coefficients. SPSS (version 14.0) was used to perform analyses; P < 0.05 was considered a significant difference.
Forty-seven boys and 44 girls with a mean ± SD age of 14.9 ± 1.1 years (range 13–16.5 years) were included. Diabetes duration was 6.4 ± 4.2 years, and A1C level was 8.8 ± 1.7% (6.2–15.0%). Of the adolescents, 19% lived in a single-parent family and 11% were from an ethnic minority. The score on the WHO-5 was 63.38 ± 18.9 (4.0–96.0).
The Flesch Reading Ease score of 90.0 indicated good readability. The CFA confirmed the one-factor structure of the WHO-5 with no significant proportion of unexplained variance [χ2 = 6.75(5), P = 0.24], an RMSEA of 0.062, and a CFI of 0.99. Cronbach's α was 0.82, and item-total correlations ranged from 0.50 to 0.72. Interitem correlations ranged from 0.33 to 0.74. The subsequent principal component analysis also suggested a unidimensional structure of the WHO-5 because the first eigenvalue was 3.1, whereas the other eigenvalues were <1.0. The explained variance was 62.5%, and the factor loading for the items ranged from 0.67 to 0.86.
As expected, higher WHO-5 scores were strongly associated with lower total CES-D and higher mental health subscale scores (r = −0.67 and 0.60, P < 0.001). More diabetes-specific family conflicts and lower self-esteem were moderately associated with lower WHO-5 scores (r = −0.34 and 0.43, respectively; P < 0.001). The positive subscale of the CES-D and the WHO-5 correlated moderately (r = 0.45, P < 0.001). WHO-5 scores were not significantly associated with the clinical variables A1C (r = −0.16, P = 0.14), diabetes duration, treatment regimen, or BMI (data not shown). WHO-5 scores did not differ for sex, age, ethnicity, or family structure.
Analyses of the ROC curves confirmed the cutoff of <50 to be optimal for detecting a mild to severe depressive affect (CES-D score ≥16) (Fig. 1). With a significant area under the curve of 0.95 (95% CI 0.91–0.99), the sensitivity was 89% with a specificity of 87%. This cutoff score identified all adolescents with CES-D ≥24, indicative of a severe depressive affect. For identification of those adolescents with an indication of a moderate to severe depressive affect, a cutoff score of 28 appeared appropriate (Table 1).
This is the first study to examine the psychometric properties of the WHO-5 in adolescents with type 1 diabetes. Our findings suggest good reliability and validity for the WHO-5, particularly given its brevity. All items were completed, and readability scores were sufficient. It would therefore seem safe to conclude that the WHO-5 is a suitable instrument to use in adolescents.
The unidimensional structure of the questionnaire was confirmed in exploratory factor analysis and CFA in our adolescent population. Further research is needed to examine the stability (test-retest) and responsiveness of the WHO-5 in adolescents. Concurrent validity was confirmed by the moderate to strong correlations with other relevant questionnaires. It would appear that the difference in time frames among the WHO-5 (past 2 weeks), the CES-D (past week), and the CHQ-CF87 (past 4 weeks) has not decreased the strength of the associations.
The WHO-5 measures (the absence of) positive affect rather than the presence of negative emotions. The strong correlation between the WHO-5 and the negatively worded items of the CES-D seems to favor the conceptualization of well-being as a continuum, with well-being and depressive affect as extremes rather than orthogonal dimensions (24). The WHO-5 was only moderately associated with the positively worded items of the CES-D. This lack of association could be due to the differences in Likert scales and item content. The WHO-5 is more focused on vigor and activity, whereas the positive subscale of the CES-D taps into feelings of self-worth and pleasure.
In our sample, those adolescents reporting low positive affect reported more negative affect on the CES-D and vice versa, confirming high sensitivity for detection of depressive affect by the WHO-5 in our sample. Although the presence of positive affect seems not to be associated with better glycemic control, more negative affect did show a moderately strong association with higher A1C levels in concert with research findings in adults (25).
We found the specificity of the WHO-5 to be relatively high, suggesting that the cutoff score of 50 as an indication for further testing for depression is appropriate for both adults and adolescents. It should be noted that only a few adolescents in our sample reported depressive symptoms, which could have resulted in relatively high sensitivity. Further research in a sample of adolescents with more depressive symptoms is therefore warranted, with comparison of WHO-5 scores to a diagnostic interview as the gold standard.
Although previous studies reported more depressive affect in girls than in boys, we did not a find such a sex effect (2,3). The difference in prevalence rates between boys and girls starts to emerge at approximately age 15 years, at which time prevalence rates of depressive affect increase in girls, whereas they remain stable in boys (26). The small sample size and relatively low number of depressed patients precluded examining the group of adolescents aged >15 years separately.
The WHO-5 has the advantage of being a generic measure of emotional well-being, allowing for comparison of well-being with that of healthy peers. Moreover, the scale can also be used to evaluate parents’ well-being and is freely available in a multitude of languages (http://www.who-5.org). However, it does not capture diabetes-specific distress for which other reliable instruments are available (e.g., the Pediatric Quality of Life Inventory Diabetes Module or the Diabetes Quality of Life-Youth) (27). In a stepped approach, the WHO-5 could be used as a first screening test alone or in combination with a diabetes-specific instrument, followed by a more extensive psychological assessment, e.g., using the Strengths and Difficulties Questionnaire (28). In summary, the WHO-5 appears to be a suitable instrument to help detect and address emotional problems in adolescents with diabetes as part of clinical routine.
Participating centers and researchers
VU University Medical Center, Amsterdam, the Netherlands: M. de Wit, MSc; Prof. H.A. Delemarre-van de Waal, PHD, MD; Prof. R.J.B.J. Gemke, PHD, MD; Prof. F.J. Snoek, PHD. Spaarne Hospital, Hoofddorp, the Netherlands: J.A. Bokma, MD. Medical Center Alkmaar, Alkmaar, the Netherlands: K. Haasnoot, MD. Juliana Kinderziekenhuis, Den Haag, the Netherlands: M.C. Houdijk, MD, PHD.
|.||≥16 (mild to severe depression)|
|.||≥24 (moderate to severe depression)|
|.||Sensitivity .||Specificity .||Sensitivity .||Specificity .|
|.||≥16 (mild to severe depression)|
|.||≥24 (moderate to severe depression)|
|.||Sensitivity .||Specificity .||Sensitivity .||Specificity .|
Data are %.
This study was supported by Dutch Diabetes Research Fund Grant 2003.00.020.
We thank all participating centers for their contribution: Medical Center Alkmaar, Spaarne Ziekenhuis Hoofddorp, and Juliana Kinderziekenhuis Den Haag.
Published ahead of print at http://care.diabetesjournals.org on 2 May 2007. DOI: 10.2337/dc07-0447.
A table elsewhere in this issue shows conventional and Système International (SI) units and conversion factors for many substances.
The costs of publication of this article were defrayed in part by the payment of page charges. This article must therefore be hereby marked “advertisement” in accordance with 18 U.S.C Section 1734 solely to indicate this fact.