Feasibility, Acceptability, and Predictive Validity of a Psychosocial Screening Program for Children and Youth Newly Diagnosed With Type 1 Diabetes

This study is a retrospective record review of all patients who were 1) admitted with a new diagnosis of type 1 diabetes at a large urban children’s hospital in the Southwestern United States during a 9-month period (July 2007–April 2008) and 2) approached to participate in a psychosocial screening assessment as part of their standard clinical care. All children with a new diagnosis of type 1 diabetes were eligible for this service; only two patients were excluded because they needed to be seen by other mental health care providers for trauma-related services unrelated to their admission for diabetes. Approval for the study was obtained from the institutional review board of record.

The initial pool included 125 children with type 1 diabetes and their caregivers. The medical records of these 125 children were reviewed for this study. Four families declined to participate in the screening. The demographics reported subsequently reflect the 121 patients whose families participated in the screening assessment; the 4 patients whose families declined are treated separately.

The sample of 121 included 64 boys and 57 girls, aged 1 to 17 years (mean, 8.8; SD, 4.2 years). More than half (58.7%) were white/Caucasian, but minority groups were also well represented: 19.0% were black/African American, 18.2% were Latino, and 4.1% were of Asian or Arabic ethnicity. Most patients (91.7%) were English speaking; the primary language in seven families (5.8%) was Spanish; and three families (2.5%) spoke Urdu or Arabic. For non-English-speaking families, the interview was completed by a psychology trainee fluent in Spanish or with the assistance of a hospital translator.

Twenty-seven patients (22.3%) lived in single-parent households. Insurance status was used as a proxy for socioeconomic status (SES). Eleven of the 121 families (9%) had no insurance, 33 (27.2%) had Medicaid, and 77 (63.6%) had private insurance. Mean HbA_1c for our sample at the time of data abstraction was 8.1% (SD, 1.4%).

A semistructured screening interview was developed to assess psychosocial factors known to put youth with type 1 diabetes at risk for poor health-related and psychologic outcomes. Following the approach delineated by Kazak et al. (12), we reviewed the research literature on early risk factors for nonadherence (1) and used other recent reviews (5) to help identify risk factors for psychologic maladjustment in response to the illness.

This literature was used to develop two semistructured interviews, one for parents of children aged 0 to 4 years and the other for parents of children aged 5 years and older. Both assessments examined risk factors in the following areas: sociodemographic and environmental factors (e.g., family structure, race/ethnicity, SES), family factors (e.g., family conflict, cohesion, and communication skills), child history of behavioral or emotional concerns, child history of developmental, neurocognitive, or learning concerns; adjustment to diagnosis (child and caregiver), and diabetes-specific concerns (e.g., needle phobia).

Interviewers were trained to ask scripted questions that could be explained or reworded as needed, although changes were generally kept to a minimum. The interview was designed so that questions did not have to be asked in a particular order, allowing the interviewer to embed questions in a more conversational framework. All participants were seen for the screening in a private room during their initial inpatient hospital admission, within 3 days of the diabetes diagnosis. At the start of the screening, the psychologist introduced himself or herself and a trainee as pediatric psychologists and behavioral health experts who are part of the diabetes team. Rights and limits of confidentiality were reviewed with the family, and written informed consent (and assent where appropriate) was obtained for the clinical visit, as was permission to release information to the medical team for continuity of care.

The trainee then completed the screening interview with the caregiver (with the child present), while the psychologist completed a brief mental status exam with the child. The child did not answer any of the interview questions. The screening interview generally took between 20 and 30 min. Within 1 to 2 weeks after discharge, the family was sent a letter summarizing the findings and recommendations.

The initial pilot data (13) were used to develop triage algorithms using risk factor counts to identify families most in need of follow-up (compared to [14]). Patients were considered for referral for psychology follow-up if they presented with six or more risk factors (i.e., >1 SD above the mean for the pilot sample), had a history of clinically significant mental health concerns, or if the caregiver requested follow-up. On the basis of these criteria, 25 of 121 children (20.6%) were referred for follow-up psychology services. Referred families were provided with a contact number in the follow-up letter and asked to contact the Psychology Service to schedule an appointment.

The total number of risk factors and each specific risk factor were both examined in our analyses. Sociodemographic and developmental risk factors were scored by their presence or absence. For example, if a caregiver was self-identified as a single parent, this was sufficient to score the risk factor as present. Family and behavioral risk factors, however, needed to meet a second criterion to be scored: a respondent who indicated the presence of behavioral risk was then directly asked to assess the current or expected affect of the factor on their child’s life. For example, if a caregiver indicated that their child had a history of behavior problems, they were asked whether this issue currently had a “significant impact” on the child’s life. Similarly, adjustment to diagnosis was screened by asking whether the child or any family members were “having a difficult time” with the diagnosis, and if so, whether this difficulty was having a “significant impact.”

Family functioning was assessed by asking caregivers to rate family conflict, communication, and closeness as average, above average (high), or below average (low); families were considered at risk if they endorsed high levels of conflict or low levels of communication or closeness between family members. Potential difficulties with diabetes management were screened by asking if the parent anticipated any difficulties with injections, blood glucose checks, or dietary changes. We did not inquire about more advanced aspects of management (e.g., carbohydrate counting) because many families were seen for screening before they had completed their diabetes education. When both parents were interviewed, a risk factor was scored if it was indicated by either parent.

To assess service utilization (ED admissions, diabetes appointment-keeping, psychology follow-up), one of us (V.D.C.) examined the appointments listed in the electronic medical record for each patient during the study period. The full records were then examined for all patients who had one of the relevant visit types to ensure that the appointment or admission occurred after the diabetes diagnosis and discharge from the hospital, was diabetes-related (in the case of ED admissions), and was not a continuation of prior treatment (in the case of psychology follow-up). A subset of these records was reviewed by a second author (D.D.S.), and all discrepancies were resolved by further reference to the medical record.

All analyses were conducted using the IBM SPSS Statistics 18 software package. α levels were set at 0.05.

During the 9-month study period, there were 161 admissions for new-onset type 1 diabetes. We approached 125 patients (77.6%) for screening. Of the 36 patients we did not approach, 21 (13.0%) were not referred to us, 13 (8.1%) were unable to be seen because of scheduling conflicts, and 2 (1.2%) were deferred because the patients needed to be seen by other mental health care providers for services unrelated to their admission for diabetes. Of the 125 families we approached, 4 (3.2%) declined to participate.

Screenings were therefore completed on 121 families, or 75.1% of the total admissions for type 1 diabetes. The interview was completed by 79 mothers (65.2%), 6 fathers (4.9%), 35 mother–father pairs (28.9%), and 1 other relative with parental permission (0.8%).

As noted, 121 of 125 families approached for the screening agreed to participate, indicating that the screening was acceptable to nearly all of the families (96.8%). Of the four families who declined, three were white, and one was of another ethnicity; all four patients were males. Two families had private insurance, one had Medicaid, and one had no insurance.

Parent satisfaction ratings for the new onset diabetes program were collected from an independent hospital survey commissioned by the Endocrine Service during a portion of the study period. A sample of 30 parents was randomly selected and interviewed by phone using a 15-question survey, with one question focused on behavioral health. Specifically, respondents were asked to “rate the effectiveness” of the behavioral health care teams, which included the Psychology Service. Nineteen parents (63.3%) rated the behavioral health service as “excellent,” seven (23.3%) rated it as “very good,” and four (13.3%) rated it as “good.” No one gave ratings of “fair” or “poor.” This resulted in a satisfaction percentage score (i.e., percent of the maximum possible average) of 90%, reflecting an average rating of 4.5 of 5 (very good to excellent).

Four of 121 children in our cohort (3.3%; three boys) had diabetes-related ED admissions in the 9 months after diagnosis. Three of the four had been referred for psychology services after the screening, although none followed up within that time period. The children were 4 years old, 5 years old, and two were 11 years old at the time of admission. Two were Latino, one was African American, and one was white. All were from English-speaking families. Two had no insurance, and two had Medicaid; none had private insurance. An independent samples t test showed no significant difference in ED admissions between children with Medicaid and children with no insurance (t = 1.437, P = 0.160); therefore, for subsequent analyses these groups were considered together.

ED admissions were positively associated with insurance status (χ² = 7.239, P = 0.007), single-parent household (χ² = 6.541, P = 0.011), caregiver with less than a high school education (χ² = 4.767, P = 0.029), parent-anticipated conflict over diabetes management (χ² = 4.128, P = 0.042), and parent-reported history of child behavior problems (χ² = 0.7.439, P = 0.006). There were no effects of age, sex, race/ethnicity, or limited English proficiency. There was also no correlation between emergency department admissions and the other outcome variables (missed diabetes clinic appointments; psychology follow-up).

A multinomial logistic regression was performed using the significant variables above to predict the likelihood of having a diabetes-related emergency department admission after diagnosis. A test of the full model versus a model with intercept only was statistically significant (χ²_{(6, 76)} = 28.569, P < 0.001). This model had a sensitivity of 75% and a specificity of 100% for predicting ED admissions. Stepwise regressions were then computed to find the best fitting model. A model using only four variables (insurance status, single-parent family, caregiver education, anticipated conflict) improved sensitivity to 100% while maintaining specificity at 98.6%.

We examined missed diabetes-related outpatient appointments (education classes and scheduled clinic visits) as an early indicator of adherence behavior (1). We did not include cancelled appointments in our analyses because we assumed that cancellations were to be expected and indicated a reasonable level of involvement in diabetes care.

In the first 9 months after diagnosis, 78 of 121 families (64.5%) did not miss an appointment, 26 (21.5%) missed one appointment, and 17 (14.0%) missed two or more appointments. Missing one or more appointments was associated with race/ethnicity (χ² = 13.331, P = 0.004), having Medicaid or no insurance (χ² = 6.313, P = 0.012), and single-parent households (χ² = 0.3.888, P = 0.049). There were no effects of age, sex, or parent education. The three significantly correlated variables were entered into a multinomial logistic regression analysis to predict families more likely to miss one or more appointments. The model was significant (χ²_{(5, 120)} = 16.223, P = 0.006), with a sensitivity of 41.9% and a specificity of 85.7%.

Pairwise comparisons revealed that African American families were significantly more likely to miss diabetes follow-up appointments than white families (P < 0.001), and were also more likely to miss appointments than Latino families, although this trend was not significant (P = 0.078). Most African American families (65.2%) missed one or more appointments, compared with 40.9% of Latino families and 23.9% of white families. Multiple appointments were missed by 43.4% of African American families, compared with 13.6% of Latino families and 4.2% of white families. Race/ethnicity remained significantly associated with missed appointments when single-parent status and insurance were held constant (F = 2.9, P = 0.038), whereas neither single-parent status nor insurance was significant when holding race/ethnicity constant. In fact, a regression model using only race/ethnicity as a predictor of missed appointment had only moderately worse sensitivity (34.9%) and slightly better specificity (89.7%) than the full model.

No associations were found between any of the predictor variables and HbA_1c.

Using the referral algorithm described in Research Design and Methods, 25 of 121 children (20.6%) were referred for follow-up psychology services, of which 13 (52%) were white, six (24%) were African American, five (20%) were Latino, and one (4%) was Arabic, which closely approximates our cohort’s demographics. Of the 25 children who were referred, only three (12%) followed up; as a result, there was only a nonsignificant trend between referral and follow-up (r = 0.166, P = 0.070). Three families who were not referred after the screening also sought psychology services at our institution. Of the six children (four girls, two boys) who did follow-up, three were Latino and three were white. No African American patients followed up for psychology services. Psychology follow-up was significantly associated with parent-reported history of child behavior problems (χ² = 3.875, P = 0.049) and prior psychologic treatment (χ² = 10.343, P = 0.001).

To our knowledge, this is the first published report of a standardized psychosocial screening for pediatric patients newly diagnosed with type 1 diabetes. We have documented that it is feasible to complete a relatively brief but comprehensive screening at diabetes diagnosis in a pediatric hospital setting. Of the 161 patients admitted for new-onset type 1 diabetes during the study period, we were able to complete screenings on 75.2%. Most of the patients who were not seen were missed due to correctable logistic factors, including not being notified of a patient’s admission or difficulties coordinating our visit with a time that the family was in the room.

Feasibility was enhanced by having the service staffed by predoctoral psychology trainees, which allowed us to provide good patient care in a relatively cost-effective manner while also contributing to the training of our students. Patients benefited from being seen by clinicians who were trained in clinical interviewing, could manage minor crises, and could provide brief on-the-spot counseling to families in greater distress. Appreciative comments from caregivers reinforced for us the value of this approach. The main limitation to relying on trainees is that this approach is only feasible at institutions with established training programs, although the screening could conceivably be implemented by other professionals with appropriate training (e.g., clinical social workers).

Our findings also indicated that the screening was highly acceptable to patients and their families. Fully 96.8% of the families we approached consented to participate. We were not able to identify any factors that discriminated families who declined from those who agreed to participate. Of note, none of the caregivers we interviewed were reluctant to answer questions about family history of mental health concerns, family stressors, their own adjustment to their child’s diagnosis, or their family’s general functioning. This argues against the concern raised by Cameron et al. (5) that assessment of caregiver and family functioning would not prove acceptable for many families. Caregiver satisfaction with the overall behavioral health service was quite high, with all 30 of the families surveyed rating the service as good to excellent.

A significant challenge to implementing the screening was the lack of a clear billing mechanism. We were able to work around this constraint through reliance on trainees and institutional support, but the longer-term feasibility of screening will depend on finding more stable ways to secure reimbursement for the service. Mental health procedure codes are not appropriate for screening because most patients would not meet criteria for a mental health diagnosis. Although we believe it would be appropriate to use a Health & Behavior assessment code (i.e., CPT 96150) given the predictive value of the screening from a medical health perspective (see below), Medicaid in our state does not cover these codes. However, it is possible that screenings in other states could be covered using these codes. Noll and Fischer (15) provide useful guidelines for advocating for the reimbursement of these codes.

In the long run, the strongest arguments for reimbursement of early screening will come from demonstrating its cost-effectiveness, which can best be accomplished by demonstrating the ability of psychosocial screening to identify patients at heightened risk for costly and avoidable complications. Our screening was able to identify factors that increased the likelihood that a child would have a diabetes-related ED admission, which can cost >$10,000 (16). In fact, a simple model using only insurance status, family structure, caregiver education level, and anticipated conflict over diabetes management had 100% sensitivity and 98.6% specificity for predicting who would or would not have an ED admission after diagnosis.

ED admissions were most likely among children living in single-parent households who had a reported history of prior behavior problems, with a nearly sixfold increase in probability of a subsequent admission after diagnosis. Preventive resources should focus on these high-risk families. For example, brief parent training in behavior management could potentially provide an empirically based and cost-effective way to reduce the likelihood of ED admissions for these children (17). Other interventions could focus on preventing conflict in families who anticipate conflict over diabetes management. Anderson et al. (18) showed that a low-cost, low-intensity psychology intervention to improve family teamwork was highly effective in preventing the typical diminishment of parent involvement seen in adolescence, helped insulate families from conflict, and resulted in improved adherence over time.

The screening was also able to identify patients at greater risk for missing diabetes clinic appointments, which is itself a risk factor for poor metabolic control (19). More than 33% of the families in our cohort missed at least one appointment within the first 9 months after diagnosis, indicating that this is a substantial problem, at least at our institution. Consistent with the literature, missed appointments were related to race/ethnicity, single-parent family status, and insurance, although race/ethnicity was by far the most important predictor. Most African American families (65%) missed one or more diabetes outpatient appointments, and 43% missed multiple appointments. In addition, no African American families followed up for recommended psychology services, despite participating in the screening and being referred for services at rates comparable to other racial/ethnic groups. Lack of follow-up among African American families appears to be part of a broader phenomenon, not just within our institution, but nationally (20). Further research is needed to help identify modifiable factors that contribute to poorer follow-up among African Americans. Reducing this gap in service utilization should be a primary goal of public health policy in the years to come.

There is now good evidence that psychologic interventions are effective in reducing nonadherent behaviors in children with chronic illness (21), but fostering follow-up for preventive services through psychology services for our cohort proved to be a general challenge. Although we were successful in identifying families who we believe could have benefited from preventive intervention—for example, three of the four children with subsequent ED admissions had been referred to psychology after the screening—very few referred families actually followed up. Poor follow-up for psychology services is not unique to our setting. For example, in an evaluation of a pediatric psychology consultation service at another large medical center, Rodrigue et al. (22) found that despite recommending follow-up for 83% of patients referred to their service, only 2% returned for outpatient psychology intervention after discharge.

Our poor follow-up rates occurred despite the high acceptability of the screening, which suggests that factors other than reticence to see a psychologist may have accounted for families not pursuing recommended care. The likeliest explanation is that the families at greatest risk are typically those with the fewest resources, which makes follow-up more difficult. This is often the “Catch-22” in prevention and intervention programs with a focus on adherence. We might also have attempted to provide a greater level of care than was needed (or perceived to be needed) by these families. For example, a “Care Ambassador”-type intervention (23,24) might have been more effective in getting the right level of care to families who did not perceive a great enough need to overcome practical barriers (such as expensive medical center parking) to follow up for appointments with our service. Finally, it is possible that families continued to associate psychology follow-up with traditional mental health interventions. Our findings did suggest that the families most likely to come for follow-up were those families with more traditional mental health concerns or previous experience with psychology services. It seems clear that for preventive health efforts through psychology to be successful, they will have to be packaged in a way that minimizes associations with mental health and instead highlights their focus on improving medical health, for example, through fostering better adherence (21).

We were only able to examine follow-up appointments with the Diabetes Clinic and the Psychology Service that occurred at our institution, and some families possibly sought services elsewhere. Our data should therefore be considered a conservative estimate of follow-up. Nonetheless, some of the relevant outpatient services provided through our hospital—including diabetes education classes and pediatric psychology services with specific expertise in diabetes—are unique to our area, making it less likely that families were obtaining these services elsewhere.

A strength of our study was the relatively large number of Latino families in our cohort, who comprised approximately 18% of our sample. As we have noted elsewhere (1), there are little published data on whether levels of risk are higher in Latino youth with type 1 diabetes. We found no significant differences between Latino and white participants on any of the outcome variables. There was also no effect of limited English proficiency, which we expected might have been a barrier for families trying to navigate our system; however, our hospital has substantial resources for monolingual Spanish-speaking families, which may have mitigated this risk.

A potential limitation of this study is the use of an interview format rather than validated questionnaires. Questionnaires have the benefit of greater reliability and portability to other sites. However, they also have significant drawbacks. Written questionnaires generally have lower completion rates than interviews, especially among higher-risk patients, many of whom have limited literacy or prefer a more personal, face-to-face approach (25) and who are therefore at risk for being disenfranchised from full participation in screening when written questionnaires are used. We therefore decided on an interview as the best way to maximize acceptability and participation in the clinical service, especially for more vulnerable patient groups.

Not all of the risk factors incorporated into our interview were related to subsequent diabetes outcomes. Our next step in designing an efficient, empirically supported screening is to refine the interview further, focusing on just those risk factors with demonstrated predictive validity. Given the short time frame so far investigated, there are important outcomes whose relationships to the screened variables remain unclear. For example, we did not find differences in glycemic control based on the results of the screening, probably because many of our patients were still in the honeymoon phase that typifies the first year after diagnosis. It will be important for future research to assess whether the screen is able to identify patients at risk for poor glycemic control later in the course of the illness.

Cameron et al. (5) recently argued that psychosocial screening should be given precedence over other routine complications screenings in diabetes care. Our article provides the first published demonstration that standardized screening of newly diagnosed patients is feasible, acceptable to families from very diverse backgrounds, and able to identify patients and families at heightened risk for early emerging nonadherent behavior and diabetes-related complications. The next step in developing an empirically supported approach to minimizing psychosocial risk in children with new-onset diabetes is to link screening to preventive intervention. We attempted to provide preventive intervention through our hospital’s Psychology Service, yet our small follow-up rates suggest that our approach to linking at-risk families (especially African American families) to preventive services is insufficient. It may be possible to improve follow-up for preventive intervention by creating programs that minimize practical barriers, highlight behavioral aspects of fostering medical health, and provide care at the level of perceived need. We believe that a program that integrates early psychosocial screening with targeted preventive intervention has substantial potential for improving the health of children and youth with type 1 diabetes.

No potential conflicts of interest relevant to this article were reported.

D.S. researched data, wrote the manuscript, and edited the manuscript. V.D.C. researched data and reviewed and edited the manuscript. M.E.A. contributed to discussion, and reviewed and edited the manuscript. B.J.A. contributed to discussion and reviewed and edited the manuscript.

The authors thank their colleagues on the Texas Children's Hospital Diabetes Inpatient-Outpatient teams whose collaboration allowed for the successful integration of this psychosocial screening protocol into the inpatient diabetes service: Stacy Ahmed, LCSW; Donnett Bell, RN; Katherine Dawn Bunting, RD; Arisally Castillo, RN; Kelley Cerasulo, RN, CPNP; Lesley Chong, MA, CCLS; Misty Davis, RN, LD; M. Kylie DeLao, RN; Matt Girotto, MHA; Morey Haymond, MD; Rubina Heptulla, MD; Anh Mac, RN; Lori Malone, MBA, MA; Kimberly Mason, RN, CDE; Sara Montenegro, MBA; Shelly Nalbone, RN; Yvette Ochoa, RN; Gail Parazynski, RN; Christina Rodriguez, RN; Jennifer Smith; Rona Sonabend, MD; Suzanne Tadlock, RN, CDE; Tangula Taylor, RN; Charlexia Taylor-McQueen, RN, CDE; Renee Thompson, MA; and Rhonda Wolfe, RN.