OBJECTIVE

Examine the utility of suicide-risk items embedded within depression screeners for identifying the presence of suicide risk in adolescents and young adults (AYA) with type 1 diabetes.

RESEARCH DESIGN AND METHODS

Sensitivity, specificity, and predictive value of self-report of suicide risk on the Patient Health Questionniaire-9 (PHQ-9) were compared with the pediatric psychologist–administered Columbia-Suicide Severity Rating Scale (C-SSRS) as the reference standard for AYA with type 1 diabetes seen in a multidisciplinary AYA Diabetes Program clinic.

RESULTS

Of 133 participants, 9.8% and 11.3% reported suicide risk on the PHQ-9 and C-SSRS, respectively. Sensitivity of the PHQ-9 risk item was 53.3% (95% CI 27.4%–77.7%), specificity was 95.7% (95% CI 89.9%–98.4%), positive predictive value was 61.5% (95% CI 32.3%–84.9%), and negative predictive value was 94.2% (95% CI 87.9–97.4%).

CONCLUSIONS

Depression screeners appear to under-identify AYA with type 1 diabetes who may otherwise be at risk for suicide.

Reliable identification of adolescents and young adults (AYA) with type 1 diabetes at risk for suicide is critical given requisite access to insulin, which can be lethal when used for self-harm (1,2). There are clear clinical guidelines for depression screening in children and adolescents with type 1 diabetes (3), yet recommendations for suicide screening are lacking (4). In this study, we examined the utility of the Patient Health Questionnaire-9 (PHQ-9) suicide-risk item for identifying the presence of suicide risk in AYA with type 1 diabetes, compared with the utility of a validated and reliable suicide-specific measure.

Participants and Procedures

AYA with type 1 diabetes seen at their first multidisciplinary AYA Diabetes Program clinic visit completed a PHQ-9 questionnaire the same day as their appointment. A pediatric psychologist administered the Columbia-Suicide Severity Rating Scale (C-SSRS), a suicide-specific interview, during the clinic visit. This study was approved by the Seattle Children’s Research Institute Institutional Review Board.

Measures

PHQ-9

Participants completed the PHQ-9, an established self-report measure used to assess symptoms of major depressive disorder (5). Participants identified frequency of thoughts about suicide or self-harm (i.e., item 9, “thoughts that you would be better off dead, or of hurting yourself in some way”) over the past 2 weeks. Item 9 is considered a nonspecific measure, encompassing both suicide and self-harm ideation (6). The PHQ-9 demonstrated good internal consistency for the present study (Cronbach α = 0.88).

C-SSRS

The Columbia-Suicide Severity Rating Scale (C-SSRS), a structured, suicide-specific interview, is used to assess passive or active suicidal thoughts, plan, and intent during the previous month (7). The C-SSRS includes distinct categories related to suicidal thoughts and behaviors that are relatively consistent with definitions outlined by the Centers for Disease Control and Prevention (8) and is considered the gold-standard suicide-specific screening instrument (9).

Data Analysis

A score of ≥1 (e.g., “several days”) on item 9 was used to indicate presence of suicide risk. Any endorsement of suicide ideation, plan, or intent in the past month was used to indicate presence of suicide risk on the C-SSRS.

Sensitivity, specificity, positive and negative predictive values, and false-positive and false-negative rates with exact 95% CIs were calculated for the PHQ-9 item 9, compared with the C-SSRS as the reference standard.

Analysis included 133 AYA with type 1 diabetes (mean age 19.6 years ± 1.1; 58% young women; Table 1). Thirteen participants (9.8%) reported suicide risk on the PHQ-9 and 15 (11.3%) screened positive on the C-SSRS. Of the 15 participants identified for suicide risk by C-SSRS, 8 screened positive for suicide risk on the PHQ-9. The PHQ-9 correctly identified the absence of suicide risk in 113 of 118 participants found not to be at risk for suicide by C-SSRS. Sensitivity of the PHQ-9 risk item compared with the C-SSRS was 53.3% (95% CI 27.4%–77.7%), specificity was 95.7% (95% CI 89.9%–98.4%), positive predictive value was 61.5% (95% CI 32.3%–84.9%), and negative predictive value was 94.2% (95% CI 87.9%–97.4%). The PHQ-9 under-identified suicide risk in 46.7% of participants (false-negative rate) and overidentified 4.2% of participants as at risk who did not report suicide risk on the C-SSRS.

Table 1

Descriptive characteristics

CharacteristicOverall sample (N = 133)Endorsed suicide risk
PHQ-9, item 9 (n = 13)C-SSRS (n = 15)
Age (years) 19.6 ± 1.1; 16, 22 20.0 ± 1.2; 17.5, 21.3 20.1 ± 1.1; 17.5, 21.3 
Female sex 68 (51.1) 8 (61.5) 9 (60.0) 
Race    
 White 99 (74.4) 9 (69.2) 11 (73.3) 
 Black 9 (6.8) 3 (23.1) 2 (13.3) 
 Asian 3 (2.2) 0 (0) 1 (6.7) 
 Multiracial 6 (4.5) 1 (7.7) 0 (0) 
 Other 12 (9.0) 0 (0) 1 (6.7) 
 Unknown/not reported 4 (3.0) 0 (0) 0 (0) 
Ethnicity    
 Non-Hispanic/Latino 116 (87.2) 13 (100.0) 15 (100.0) 
 Hispanic/Latino 12 (9.0) 0 (0) 0 (0) 
 Unknown/not reported 5 (3.8) 0 (0) 0 (0) 
Preferred language of care    
 English 125 (94.0) 13 (100.0) 14 (93.3) 
 Spanish 1 (0.8) 0 (0) 0 (0) 
 Other 3 (2.2) 0 (0) 1 (6.7) 
 Unknown/not reported 4 (3.0) 0 (0) 0 (0) 
Type 1 diabetes characteristics    
 Age at diagnosis (years) 9.8 ± 4.3; 0.5, 18.5 9.1 ± 5.2; 0.5, 15.6 9.5 ± 4.9; 1.4, 16.4 
 Duration (years) 9.7 ± 4.3, 1.0, 19.7 10.8 ± 5.0; 4.4, 20.0 10.6 ± 4.9; 1.8, 19.4 
Insulin delivery method    
 Multiple daily injections 56 (42.1) 8 (61.5) 8 (53.3) 
 Pump 74 (55.6) 5 (38.5) 6 (40.0) 
 CGM prescription 75 (56.4) 5 (38.5) 7 (46.7) 
HbA1c 8.7 ± 2.0; 5.4, <14 9.8 ± 2.4; 7.0, 13.9 9.3 ± 1.9; 6.9, 13.2 
CharacteristicOverall sample (N = 133)Endorsed suicide risk
PHQ-9, item 9 (n = 13)C-SSRS (n = 15)
Age (years) 19.6 ± 1.1; 16, 22 20.0 ± 1.2; 17.5, 21.3 20.1 ± 1.1; 17.5, 21.3 
Female sex 68 (51.1) 8 (61.5) 9 (60.0) 
Race    
 White 99 (74.4) 9 (69.2) 11 (73.3) 
 Black 9 (6.8) 3 (23.1) 2 (13.3) 
 Asian 3 (2.2) 0 (0) 1 (6.7) 
 Multiracial 6 (4.5) 1 (7.7) 0 (0) 
 Other 12 (9.0) 0 (0) 1 (6.7) 
 Unknown/not reported 4 (3.0) 0 (0) 0 (0) 
Ethnicity    
 Non-Hispanic/Latino 116 (87.2) 13 (100.0) 15 (100.0) 
 Hispanic/Latino 12 (9.0) 0 (0) 0 (0) 
 Unknown/not reported 5 (3.8) 0 (0) 0 (0) 
Preferred language of care    
 English 125 (94.0) 13 (100.0) 14 (93.3) 
 Spanish 1 (0.8) 0 (0) 0 (0) 
 Other 3 (2.2) 0 (0) 1 (6.7) 
 Unknown/not reported 4 (3.0) 0 (0) 0 (0) 
Type 1 diabetes characteristics    
 Age at diagnosis (years) 9.8 ± 4.3; 0.5, 18.5 9.1 ± 5.2; 0.5, 15.6 9.5 ± 4.9; 1.4, 16.4 
 Duration (years) 9.7 ± 4.3, 1.0, 19.7 10.8 ± 5.0; 4.4, 20.0 10.6 ± 4.9; 1.8, 19.4 
Insulin delivery method    
 Multiple daily injections 56 (42.1) 8 (61.5) 8 (53.3) 
 Pump 74 (55.6) 5 (38.5) 6 (40.0) 
 CGM prescription 75 (56.4) 5 (38.5) 7 (46.7) 
HbA1c 8.7 ± 2.0; 5.4, <14 9.8 ± 2.4; 7.0, 13.9 9.3 ± 1.9; 6.9, 13.2 

Data are reported as n (%) or mean ± SD; minimum, maximum. CGM, continuous glucose monitor.

To align with clinical guidelines, numerous clinical sites have implemented routine depression screening in AYA with type 1 diabetes, many of which are also used to screen for suicide risk (1,2,4,10). Results of this study suggest sole reliance on suicide-risk items on depression screeners may not be enough to reliably identify patients at risk for suicide. In the present study, the PHQ-9 failed to detect nearly 50% of patients who reported suicide risk on the C-SSRS. These findings are consistent with those of Horowitz et al. (11) who found low sensitivity (70%) for item 9 on the PHQ-A (i.e., the PHQ-9 adapted for adolescents) in detecting suicide risk in pediatric medical inpatients.

Observed rates of suicide risk in our study (11.3%) are among the highest reported in the pediatric diabetes literature (range 2.2%–8.9%), though rates as high as 21% have been reported when history of suicide attempt was included as a suicide risk factor. In a recent study by Majidi et al. (2), nearly 9% pediatric patients with type 1 diabetes (aged >10 years) endorsed the suicide risk item on the PHQ-9. Our findings suggest integration of a suicide-specific screener could have resulted in the identification of additional patients at risk for suicide.

Differences in observed rates of suicide risk across the PHQ-9 and C-SSRS may be associated with the nature of questions on each measure. The PHQ-9 suicide-risk item may perform poorly because it does not directly ask about thoughts of killing oneself (11) nor does it delineate between passive thoughts about death and self-harm ideation. It uses vague phrasing (e.g., lack of clarity about what “several” or “bothered by” means) and does not assess other important suicide risk factors (e.g., active suicide ideation, plan, or intent) that are considered essential elements of suicide risk assessment (12). The extent to which the second half of the question (i.e., “hurting yourself in some way”) may reflect thoughts of self-harm versus other behaviors, such as nonadherence, is unclear (13). Higher rates of suicide risk reported on the C-SSRS may be associated with the inclusion of direct questions about passive and active suicide ideation, plan, and intent. The C-SSRS may have identified more patients because of the longer assessment window (1 month) compared with the PHQ-9 (2 weeks).

Although depression screeners play an important role in the care of youth with type 1 diabetes, results of this study suggest integration of suicide-specific measures may aid in the identification of those at risk for suicide and minimize the number of patients who may move through the health care system undetected (11). Suicide screeners, such as the Ask Suicide-Screening Questions, can be used to triage services, including quickly identifying patients requiring additional assessment or intervention related to suicidality (e.g., referral to therapy, emergency services in the event of imminent risk). The Ask Suicide-Screening Questions can be integrated into standard clinical practice, administered by nonmental health personnel (14), and takes <20 s to complete (15), thus making it a potentially excellent option for suicide-specific screening in an ambulatory clinic setting with limited behavioral health resources.

This study’s key strengths are its use of a suicide-specific instrument to assess suicidal thoughts and behaviors and use of a clinically derived sample. We are unable to discern how suicide risk may change over time and caution extending our findings to younger patients. In addition, although it is possible there was underreporting of suicide risk in the in-person C-SSRS interview format, it does not alter the finding that the PHQ-9 failed to detect nearly 50% of patients who reported suicide risk on the C-SSRS. Finally, it is unclear to what extent the presence of a pediatric psychologist may have influenced intake practices, particularly for patients with greater mental health needs, and thus we cannot exclude referral bias.

Given mounting evidence of suicide risk in this population, clinical guidelines should explicitly recommend use of validated, suicide-specific screeners in addition to routine depression screening as a part of comprehensive medical care for this vulnerable population. The findings of this study also reinforce the critical role of embedded behavioral health resources as part of a multidisciplinary team to address the needs of this unique population.

See accompanying article, p. 1035.

Acknowledgments. The authors acknowledge the Forest and Sequoia Foundations for their support of the clinical programming and care provided at the University of Washington and Seattle Children’s Hospital Adolescent and Young Adult Diabetes Program.

Funding. F.S.M.’s time was supported by a K23 Career Development Award from the National Institute of Diabetes and Digestive and Kidney Diseases of the National Institutes of Health (grant DK119465).

Duality of Interest. No potential conflicts of interest relevant to this article were reported.

Author Contributions. A.C.M. conceptualized and designed the study, completed data acquisition, contributed to the analysis plan and interpretation, drafted the initial manuscript, and is the guarantor of this work and, as such, had full access to all the data in the study and takes responsibility for the integrity of data and the accuracy of data analysis. A.J.R., C.E.T., K.W.W., C.P., and I.B.H. contributed to manuscript preparation and review. J.R.Y.-F. contributed to data interpretation and manuscript preparation and review. K.L.R. contributed to data collection and manuscript preparation and review. A.C.M. is the guarantor of this work and, as such, had full access to all the data in the study and takes responsibility for the integrity of data and the accuracy of data analysis.

Presentation. Data from this study were presented at the 2020 virtual American Diabetes Association Scientific Sessions.

1.
Corathers
S
,
Mara
CA
,
Chundi
PK
,
Kichler
JC
.
Depression screening of adolescents with diabetes: 5-years of implementation and outcomes
.
J Am Acad Child Adolesc Psychiatry
2019
;
58
:
628
632
2.
Majidi
S
,
O’Donnell
HK
,
Stanek
K
,
Youngkin
E
,
Gomer
T
,
Driscoll
KA
.
Suicide risk assessment in youth and young adults with type 1 diabetes
.
Diabetes Care
2020
;
43
:
343
348
3.
American Diabetes Association
.
13. Children and adolescents: standards of medical care in diabetes-2021
.
Diabetes Care
2021
;
44
(
Suppl. 1
):
S180
S199
4.
Matlock
KA
,
Yayah Jones
NH
,
Corathers
SD
,
Kichler
JC
.
Clinical and psychosocial factors associated with suicidal ideation in adolescents with type 1 diabetes
.
J Adolesc Health
2017
;
61
:
471
477
5.
Kroenke
K
,
Spitzer
RL
,
Williams
JB
.
The PHQ-9: validity of a brief depression severity measure
.
J Gen Intern Med
2001
;
16
:
606
613
6.
Thom
R
,
Hogan
C
,
Hazen
E
.
Suicide risk screening in the hospital setting: a review of brief validated tools
.
Psychosomatics
2020
;
61
:
1
7
7.
Posner
K
,
Brent
D
,
Lucas
C
, et al
.
Columbia-Suicide Severity Rating Scale (C-SSRS)
.
2008
.
8.
Crosby
AE
,
Ortega
L
,
Melanson
C
.
Self-Directed Violence Surveillance: Uniform Definitions and Recommended Data Elements, Version 1.0
.
Atlanta, GA
,
Centers for Disease Control and Prevention
;
2011
9.
Giddens
JM
,
Sheehan
KH
,
Sheehan
DV
.
The Columbia-Suicide Severity Rating Scale (C-SSRS): has the “gold standard” become a liability?
Innov Clin Neurosci
2014
;
11
:
66
80
10.
Mulvaney
SA
,
Mara
CA
,
Kichler
JC
, et al
.
A retrospective multisite examination of depression screening practices, scores, and correlates in pediatric diabetes care
.
Transl Behav Med
2021
;
11
:
122
131
11.
Horowitz
LM
,
Mournet
AM
,
Lanzillo
E
, et al
.
Screening pediatric medical patients for suicide risk: is depression screening enough?
J Adolesc Health
2021
;
68
:
1183
1188
12.
Posner
K
,
Melvin
GA
,
Stanley
B
,
Oquendo
MA
,
Gould
M
.
Factors in the assessment of suicidality in youth
.
CNS Spectr
2007
;
12
:
156
162
13.
Thompson
R
,
Henkel
V
,
Coyne
JC
.
Suicidal ideation in primary care: ask a vague question, get a confusing answer
.
Psychosom Med
2004
;
66
:
455
456
14.
Ballard
ED
,
Cwik
M
,
Van Eck
K
, et al
.
Identification of at-risk youth by suicide screening in a pediatric emergency department
.
Prev Sci
2017
;
18
:
174
182
15.
Sullivant
SA
,
Bradley-Ewing
A
,
Williams
DD
, et al
.
Prevalence of positive suicide risk screens among adolescents with type 1 diabetes (T1D)
.
J Psychosom Res
2020
;
138
:
110247
Readers may use this article as long as the work is properly cited, the use is educational and not for profit, and the work is not altered. More information is available at https://www.diabetesjournals.org/journals/pages/license.