Measuring the Stigma Surrounding Type 2 Diabetes: Development and Validation of the Type 2 Diabetes Stigma Assessment Scale (DSAS-2)

For the person with type 2 diabetes, managing the condition is a relentless activity. This not only places an emotional burden on the individual (1) but may also invite negative attention, comment, and judgment from others, suggesting a socioemotional burden. Whereas our previous review demonstrates that people without diabetes do not perceive the condition as stigmatized (2), there is now a reasonable body of evidence to indicate that people with type 2 diabetes perceive and experience stigma related to their condition. Qualitative research has provided a rich description of diabetes stigma from the perspective of adults with type 2 diabetes (3–6). They feel blamed and judged by others (e.g., for “bringing it on themselves”) and report being subject to negative stereotyping (e.g., fat, lazy), leading to exclusion, rejection, discrimination, and restricted/lost life opportunities (6).

The multinational Diabetes Attitudes, Wishes, and Needs second study (DAWN-2) corroborates these qualitative findings, with one in five (19.2%) respondents reporting being discriminated against (7). However, in this study, discrimination (just one specific manifestation of stigmatization) was assessed using a single item, and so we remain unable to quantify the full extent and experience of diabetes stigma. The International Diabetes Federation has identified diabetes stigma as a problem that needs urgent attention, with a key priority being to “champion a world free from discrimination and stigma for people with diabetes” (8). However, before we can fully address diabetes stigma at the individual, environmental, and societal levels, we need research to investigate the following: the extent and severity of the problem, its determinants and consequences (e.g., for emotional well-being, diabetes self-management, and clinical outcomes), and changes over time or in response to intervention (9).

Rigorous research cannot take place without a validated, self-report measure of diabetes stigma. Previous studies have used generic measures of health-related stigma (10,11) or unvalidated diabetes-specific questions (7,12). Although valid and appropriate for use in some contexts, generic measures do not capture the nuances of negative social judgment and stereotyping that are associated specifically with type 2 diabetes, and some include content that is irrelevant or inappropriate for people with type 2 diabetes (e.g., about fear of touching due to contagion). Study-specific or ad hoc approaches to quantifying diabetes stigma are typically unsystematic and unvalidated and make comparisons across studies impossible due to variations in operationalization and measurement. Thus, our aim was to develop and validate a self-report measure of perceived and experienced stigma for use with people with type 2 diabetes: the Type 2 Diabetes Stigma Assessment Scale (DSAS-2).

All study activities were approved by the Deakin University Human Research Ethics Committee (2011-046).

The content of the questionnaire was informed by qualitative data about diabetes stigma from 25 adults with type 2 diabetes, published elsewhere (6). One author drafted an initial 125-item pool based on participant quotes from interview transcripts, representing all key themes in the data. The items were constructed as statements with a 5-point Likert response scale (strongly disagree to strongly agree). An additional 30 items were drafted based on items used in health-related stigma scales (e.g., Weight Self-Stigma Scale [13] and Chronic Illness Anticipated Stigma Scale [11]).

The 155-item pool was reduced to 91 items through review by all authors, with consensus about item redundancy and revisions reached through discussion. Four external experts (in health psychology, health stigma, and/or diabetes) were invited to review the 91 items. Feedback was collated and reviewed by all authors, who made the final decisions about item revisions and deletions. A total of 57 items were taken forward into the cognitive debriefing phase.

Cognitive debriefing interviews were conducted to assess the clarity, relevance, and comprehensiveness of the questionnaire instructions, items, and response options from the perspective of adults with type 2 diabetes. Participants were recruited through advertisements in diabetes media (e.g., e-newsletters) and social media (e.g., Facebook pages). Upon making contact with researchers, potential participants were screened for eligibility (self-reported type 2 diabetes diagnosis, aged 18–75 years, fluent in English, able to access/use Internet, and living in Australia) and asked to provide written informed consent. Then a telephone interview was scheduled. Participants were then directed to a website and requested to complete an online version of the stigma questionnaire while making notes for discussion on a paper copy (sent to them by post), in the 24 h prior to their scheduled interview. Interviews lasted an average of 28 min (range 15–45). Participants were entered into a prize draw to win a year’s supply of (12) movie tickets.

An online survey was conducted to collect response data on the type 2 diabetes stigma item pool, and a selection of previously validated questionnaires (see Materials). The website featured a plain language description of the study, a consent form, eligibility screening, and the online survey (securely hosted by Qualtrics, Provo, UT). Participants who did not have access to the internet could contact researchers to request a hard-copy survey.

Eligible participants were English-speaking adults with type 2 diabetes, aged 18–75 years, living in Australia. An invitation letter was mailed to a random sample of 12,000 National Diabetes Services Scheme (NDSS) registrants with type 2 diabetes, stratified by insulin use (50% insulin treated, intentionally oversampled) and by state of residence. The letter introduced the study and directed people to the study website. The study was also advertised nationwide in diabetes-related media (e.g., e-newsletters and social media). Twenty-five people requested and returned hard copy versions of the consent form and survey. Participants were entered into a prize draw to win one of three iPad Minis.

The survey included the diabetes stigma item pool as well as a selection of psychosocial measures for validation purposes and questions about demographic and clinical characteristics. Table 1 summarizes the types of validity assessed and measures used.

The Stigma Scale for Chronic Illness 8-item scale (SSCI-8) (14), a generic scale that assesses internalized and enacted stigma for chronic conditions, was used to assess concurrent validity. It comprises eight statements (e.g., “Because of my condition, people were unkind to me”) and a 5-point Likert response scale (1 = never to 5 = always). Item scores are summed to form a total score (range 8–40), where higher scores indicate more internalized/enacted stigma. Positive correlations were expected with the DSAS-2.

Convergent validity was assessed against the following.

The Patient Health Questionnaire 8-item scale (PHQ-8) and Generalized Anxiety Disorder 7-item scale (GAD-7) (15,16) assess depressive and anxiety symptomatology, respectively, by asking respondents to indicate how frequently they have experienced eight symptoms of depression (e.g., “little interest or pleasure in doing things”) and seven symptoms of anxiety (e.g., “feeling afraid as if something awful might happen”) over the last 2 weeks. For both scales, items are scored on a 4-point Likert scale (0 = not at all to 3 = nearly every day) and summed to produce a total score (PHQ-8 range 0–24; GAD-7 range 0–21). Higher scores indicate more depressive/anxiety symptoms. Positive correlations were expected with the DSAS-2.

Problem Areas in Diabetes (PAID) scale (17) is a 20-item measure of diabetes-specific distress. Respondents indicate how much of a problem each statement (e.g., “feeling ‘burned out’ by the constant effort needed to manage diabetes”) is for them on a 5-point Likert scale (0 = not a problem to 4 = serious problem). Items are summed and converted to a percentage score, with higher scores indicating greater diabetes distress. A positive correlation was expected with the DSAS-2.

Rosenberg Self-esteem scale (RSE) (18) is a 10-item measure of self-esteem. Respondents rate statements (e.g., “I take a positive attitude toward myself”) on a 4-point Guttman scale (0 = strongly disagree to 3 = strongly agree). A total score (range 0–30) is calculated by summing responses to individual items, after reverse scoring negatively worded items. Higher scores indicate higher self-esteem. A negative correlation was expected with the DSAS-2.

Discriminant validity was assessed against diabetes duration (years) and scores on the Prospective and Retrospective Memory Questionnaire (PRMQ) (19). The PRMQ assesses general prospective and retrospective memory. Respondents indicate how frequently they experience 16 different memory “slips” (e.g., “Do you fail to mention or give something to a visitor that you were asked to pass on?”) by responding on a 5-point Likert scale (1 = never to 5 = very often). Responses are summed to produce a total score (range 16–80); higher scores indicate more memory problems.

All the validated scales demonstrated high internal consistency reliability in the current sample (all α ≥0.90, range 0.90–0.96). The measures were ordered in the survey in such a way as to minimize priming 1) clinical characteristics, 2) PHQ-8, 3) GAD-7, 4) PAID, 5) diabetes stigma items, 6) SSCI-8, 7) RSE, and 8) demographics.

Kolmogorov-Smirnov tests demonstrated that all scale data had nonnormal distributions, necessitating the use of nonparametric statistics.

Data on all scales were not missing completely at random (Little MCAR test, all P < 0.05); however, missing data patterns did not differ by any key demographic, clinical, or psychosocial factors, suggesting that imputation was appropriate. Where respondents had ≤10% of data points missing on a measure, their missing data were imputed using expectation maximization techniques (20) to facilitate calculation of a total score. Total scores were not calculated when >10% of data points were missing. No missing data were tolerated for the calculation of subscale scores. High overall completion rates (≥90%) were taken as evidence of acceptability.

Prior to proceeding with dimension reduction analyses, items with a large number (five or more) of very high (r_s >0.7) or very low (r_s <0.3) inter-item correlations were screened out using two-tailed inter-item Spearman ρ correlations. Participants with missing data were deleted pairwise, minimizing loss of data by using all available cases for each analysis.

A Kaiser-Meyer-Olkin statistic of 0.98 indicates that our sample size was more than adequate to proceed with dimension reduction analyses (21). The structural validity of the questionnaire was assessed using principal components analysis with direct oblimin rotation. Decisions about how many factors to retain were informed by Kaiser eigenvalue (≥1), percentage variance explained by each factor, and factor loadings. Modern guidelines suggest that multiple factor loadings of ≥0.5 indicate likely strong and stable factors, and thus this cutoff was used to identify meaningful loadings (22). Cronbach α was used to assess internal consistency reliability of factors (subscales) and a forced one-factor solution (total scale).

Item scores were summed to produce subscale and scale total scores. Participant response patterns on the final scale were examined using descriptive statistics.

Concurrent, convergent, and discriminant validity were assessed using two-tailed Spearman ρ correlations. Participants with missing total scores were deleted pairwise, minimizing loss of data by using all available cases for each analysis. In accordance with Cohen’s guidelines, medium-to-large correlations (r_s >±0.4) were taken as evidence for concurrent and convergent validity, and small correlations (r_s <±0.3) were taken as evidence of discriminant validity (23).

All statistical analyses were conducted using IBM SPSS for Mac, version 22 (Chicago, IL). An α level of P < 0.05 was taken to indicate significance.

Thirteen adults with type 2 diabetes (median age 60 years [range 39–75]) took part in cognitive debriefing interviews. All had at least some high school education; a minority (n = 4, 31%) had a university degree; two (15%) were born outside Australia; and five (38%) participants used insulin.

Participants indicated that the questionnaire items were easy to understand, relevant, and comprehensive. Participants demonstrated a clear and accurate understanding of the word “stigma.” Iterative changes to the questionnaire were made throughout the cognitive debriefing phase if consistent feedback was received from three or more participants. Examples include the following: deletion of redundant items, changing the middle response option from “neither disagree nor agree” to “unsure,” and adding additional questionnaire instructions.

Where feedback was inconsistent between participants, no immediate action was taken, and instead all authors reviewed these outstanding issues and achieved consensus regarding revisions through discussion. This process resulted in a reduced and revised type 2 diabetes stigma item pool of 48 items (down from 57), each with a 5-point Likert response scale (1 = strongly disagree, 2 = disagree, 3 = unsure, 4 = agree, and 5 = strongly agree).

A total of 1,264 eligible adults with type 2 diabetes participated. Of those, 1,064 completed the survey section that included the diabetes stigma items (n = 200 discontinued prior to this). Thus, analyses were performed on this sample of n = 1,064 and their characteristics are displayed in Table 2.

A total of 989 (93.0%) respondents completed all 48 diabetes stigma items (i.e., no missing data), indicating a high level of acceptability, especially given the large number of items and relatively sensitive content. Forty-three (4.0%) respondents had one missing data point, 11 (1.0%) had two missing data points, and 21 (2.0%) had more than two missing data points (range 3–48).

Very high multicollinearity was observed among the 48 items (determinant = 7.00^e−19). Screening of inter-item correlations resulted in deletion of eight items (three due to high correlations and five due to low correlations).

An unforced principal components analysis performed on the remaining 40 items revealed four factors (eigenvalue ≥1). However, the fourth factor had a low eigenvalue (1.14), explained little variance (2.9%), and comprised just one item. A forced three-factor solution revealed three potential subscales best described as Treated Differently (11 items), Blame and Judgment (12 items), and Self-stigma (7 items). Ten items did not load on any factor using the ≥0.5 criteria and were therefore excluded from further analyses.

Cronbach α coefficients for the three subscales were high (all ≥0.92), indicating potential redundancy, so further scale reduction was undertaken by reviewing item content, inter-item and item-total correlations, and α coefficients. Scale reduction decisions for each subscale were as follows:

• Treated Differently: Five items were removed (two due to redundant content and numerous high inter-item correlations and three due to content not being well aligned with other items in the subscale).

• Blame and Judgment: Six items were removed (three due to redundancy and multiple, very high inter-item and item-total correlations and three that asked participants to compare stigma associated with type 1 and type 2 diabetes on the basis of concerns that they may serve to perpetuate negative feelings/attitudes toward other groups). One item (previously dropped from the scale on the basis that it did not load ≥0.5) was reinstated, as it uniquely represented the experience of feeling blamed for “bringing diabetes on myself,” a key theme identified from the qualitative interviews (6).

• Self-stigma: One item was removed because it was almost identical to another item but used more complicated language.

An unforced three-factor solution comprising the final 19 items is presented in Table 3, along with a forced one-factor solution, supporting calculation of a total scale score. In the three-factor solution, one item (“I feel embarrassed in social situations because of my type 2 diabetes”) double loaded on the Treated Differently and Self-stigma factors. However, it loaded on the latter marginally more strongly, and the wording was considered more relevant to the self-stigma construct; thus, it was included in the scoring of the latter subscale only.

The three-factor and one-factor solutions were stable, with findings being replicated on two randomly selected subsamples (n = 518 and n = 546).

The final scale had a Flesch-Kincaid reading ease score of 69.7, suggesting that the scale is constructed in plain English that is readable for people with some secondary school education (8th to 9th grade reading level).

The full range of response options was used for every item, although for most, the distribution of responses was skewed positively (Table 4). Missing data ranged from 1.2 to 1.9% per item.

The mean ± SD Total Diabetes Stigma score was 41.0 ± 15.9 (range 19–90); median was 40.0 (27, 52). Subscale scores were as follows: Treated Differently: 11.0 ± 4.8 (range 6–30), 10 (6, 14); Blame and Judgment: 18.1 ± 7.2 (range 7–35), 18 (12, 24); and Self-stigma: 11.9 ± 5.8 (range 6–30), 11 (7, 15). A total of 205 (19.3%) respondents scored more than one SD above the mean total diabetes stigma score, suggestive of potentially problematic perceived and experienced diabetes stigma.

Satisfactory concurrent validity was demonstrated by a large positive correlation between the DSAS-2 and the SSCI-8 (r_s = 0.78, P < 0.001). Convergent validity was confirmed with a strong positive correlation with the PAID (r_s = 0.67, P < 0.001), medium positive associations with the PHQ-8 and GAD-7 (r_s = 0.44 and 0.46, respectively, both P < 0.001), and a medium negative correlation with the RSE (r_s = −0.51, P < 0.001). Discriminant validity was confirmed as the DSAS-2 had weak(er) associations with the PRMQ (r_s = 0.28, P < 0.001) and diabetes duration (r_s = −0.04, NS).

The DSAS-2 meets the need for a self-report measure of type 2 diabetes stigma, which can now be used in research to quantify the extent and impact of stigma, as perceived and experienced by adults with type 2 diabetes. This novel measure has a comprehensive yet simple three-factor structure, which encapsulates both enacted and self-stigma, consistent with established theoretical perspectives (2,9,24,25). Importantly, the measure can also be reduced to a single dimension, facilitating the calculation of a total scale score. Our scale development and validation process has resulted in an acceptable, relatively brief, and usable measure of diabetes stigma, with strong psychometric properties.

Small proportions of missing data and full use of response options point to the acceptability of the scale. The 19-item scale is comparable in length to other widely used diabetes-specific measures, such as the PAID (17) and the Insulin Treatment Appraisal Scale (26). The DSAS-2 scale and subscales all have high internal consistency reliability, and the measure has good concurrent, convergent, and discriminant validity. Although the discriminant validity analyses resulted in a statistically significant correlation between the DSAS-2 and PRMQ scores (likely due to the large sample size), the effect size was small, indicating a negligible relationship between the variables. As diabetes stigma is likely to have pervasive cognitive, emotional, and social impacts, we anticipated that many constructs would have some relationship, however small, with diabetes stigma. Diabetes duration and the PRMQ were considered good candidates for assessing discriminant validity of the DSAS-2 because we made a priori predictions that these relationships would be small, negligible, or indirect; thus discriminant validity of the DSAS-2 was confirmed (27).

The Cronbach α for the full 19-item scale is high (0.95), suggesting further scale reduction might be possible. However, we believe that further item deletion will reduce the comprehensiveness and face validity of the scale, and reduce the internal consistency reliability of the subscales.

The item “I feel embarrassed in social situations because of my type 2 diabetes” double loaded on the Treated Differently and Self-stigma factors (but was only used for calculation of the latter subscale). For future use, we recommend removing the phrase “in social situations” from this item, as this will further differentiate it from the concept of being treated differently by others and will keep the focus on self-perception.

A key strength of this study is the rigorous process used to design, debrief, and validate the DSAS-2. Content for the questionnaire was developed directly from the themes that emerged from a qualitative study of stigma with 25 adults with type 2 diabetes (6). The stigma item pool was debriefed in full with a further 13 people with type 2 diabetes, and as such, adults with type 2 diabetes were fully engaged as key informants during questionnaire development. Survey participants constituted a national sample of adults with type 2 diabetes, and a wide range of ages and socioeconomic backgrounds were represented.

It is likely that the sample constitutes a representative spread of views about diabetes stigma among Australians with type 1 or type 2 diabetes, as substantial efforts were made to reduce recruitment bias in this regard. The true purpose of the study (i.e., “diabetes stigma”) was not disclosed but rather it was advertised as a survey about what it is like to live with diabetes. Furthermore, the full survey included many other questions about psychosocial aspects of diabetes, so the advertised purpose was not misleading.

This study also has a number of limitations. Although the primary recruitment method for study 2 involved sending invitations to a national random sample of NDSS registrants, participation was voluntary and thus the sample (while large) is self-selected and potentially biased, although not in relation to their experience of social stigma as the true objective was concealed during recruitment. The sample may be more likely to have high health literacy or be more engaged in issues of diabetes management than the general population of Australian adults with diabetes. The sample appears mostly representative of Australian adults with type 2 diabetes, although the proportion who use insulin is higher here (42 vs. 24% [28]) due to intentional oversampling. The questionnaire was developed and made available in English only, due to funding constraints, which may explain the relative lack of ethnic diversity in the sample. Finally, we were unable to assess responsiveness or predictive validity in the current study, and these aspects of psychometric validation will need to be established in future work.

The next steps in our research program will be to 1) quantitatively assess the relationships between diabetes stigma and other outcomes of interest, e.g., psychological well-being and self-management, and 2) develop a comparable self-report measure of perceived and experienced stigma for adults with type 1 diabetes (work ongoing). Future research will need to confirm the psychometric properties of the DSAS-2 in other samples. As the DSAS-2 has been developed and validated with an Australian sample, it will be important to debrief the measure with English-speaking participants from other countries and cultures and for appropriate cultural adaptation and translation techniques to be used to enable use in non–English-speaking populations.

Our findings indicate that the DSAS-2 is a reliable and valid measure of perceived and experienced stigma for use with adults with type 2 diabetes. The DSAS-2 fulfills the need for a validated, diabetes-specific measure of stigma and will facilitate new research in the field to further our understanding of the determinants, extent, and impact of diabetes stigma.

Acknowledgments. The authors thank Prof. Jorg Huber (University of Brighton), Dr. Giesje Nefs (Tilburg University), Prof. Frans Pouwer (Tilburg University), and Dr. Rebecca Puhl (University of Connecticut Rudd Center for Food Policy and Obesity) for their expert review of the draft diabetes stigma item pool. The authors also thank Shaira Baptista and Elizabeth Holmes-Truscott (The Australian Centre for Behavioural Research in Diabetes) for their assistance in setting up the online survey and data cleaning. Finally, the authors thank all of the study participants (debriefing and online survey) for volunteering their time, insights, and experiences.

The DSAS-2 is the copyright of The Australian Centre for Behavioural Research in Diabetes, a partnership for better health between Diabetes Victoria and Deakin University. It is available free of charge to academic researchers, clinicians, and students for use in non–commercially funded research. For commercially funded studies (initiated or sponsored by industry), a license fee will apply. Potential users are advised to email [email protected] to enquire about or access the latest version of the questionnaire and scoring guidance.

Funding. The design and development of the questionnaire was funded by a Diabetes Australia Research Trust grant.

Duality of Interest. Enhancement of the sample size for the validation study was enabled by an unrestricted educational grant from Sanofi. No other potential conflicts of interest relevant to this article were reported.

Author Contributions. J.L.B. conceptualized the study, developed the initial diabetes stigma item pool, conducted the data collection and analyses, and wrote the first draft of the manuscript. A.D.V. provided substantive intellectual input into the study protocol, reviewed and revised the DSAS-2, conducted the data collection and analyses, and provided feedback and revisions on the first and subsequent drafts of the manuscript. K.M. provided substantive intellectual input into the study protocol, reviewed and revised the DSAS-2, reviewed the data collection and analyses for the purposes of making decisions about scale reduction, and provided feedback and revisions on the first and subsequent drafts of the manuscript. J.S. conceptualized the study, reviewed and revised the DSAS-2, reviewed the data collection and analyses for the purposes of making decisions about scale reduction, and provided feedback and revisions on the first and subsequent drafts of the manuscript. All authors approved the final submitted manuscript. J.L.B. is the guarantor of this work and, as such, had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Prior Presentation. Parts of this study were presented in poster form at the 76th Scientific Sessions of the American Diabetes Association, New Orleans, LA, 10–14 June 2016.