Evaluation of Diabetes and Cardiovascular Disease Print Patient Education Materials for Use With Low–Health Literate Populations

ADA and AHA patient education materials meeting the following criteria were sought for inclusion in the study: print format, content addressing behavioral management of diabetes and cardiovascular disease (CVD) or CVD risk in individuals with diabetes, material written for an adult audience, and English language. Materials in the Diabetes & CVD Toolkit, comprising patient education brochures designed for health professionals to distribute to their patients, were obtained electronically from the ADA Web site (http://www.diabetes.org/for-health-professionals-and-scientists/CVD.jsp). Twenty-one of 26 items met inclusion criteria. Reasons for exclusion were as follows: content did not explicitly focus on diabetes and CVD self-management behaviors (e.g., prediabetes, depression) (n = 3) or item was a log sheet rather than an educational brochure (n = 2).

Requests for each of 44 titles were placed via the AHA Web site (http://www.americanheart.org/presenter.jhtml?identifier=1200021) for mailing to the investigators. Eighteen requested titles were received. An additional six titles that were not found on the website were also received. Nineteen of these 24 materials met inclusion criteria. Five were excluded due to focus on a specific cardiac concern but without a direct diabetes focus in the text (e.g., bypass surgery, congenital heart disease, cardiac catherization, managing angina, and recovery from heart attack).

Materials were evaluated using 32 consensus criteria from widely cited authoritative sources on development and evaluation of low literacy and understandable consumer health information (6–8). Criteria addressing literacy demand (characteristics that impact approachability, readability, processing, and comprehension of print information) and behavioral activation (characteristics that impact meaningfulness, application, and usability of print information) were compiled.

Entire document review, including page-by-page and item-by-item analyses, was used to evaluate most criteria. For reading level, use of active voice, and sentence, paragraph, and line length, a sampling method described by Doak et al. was used (7). Three samples of text (from the beginning, middle, and end), totaling ∼30 sentences, were systematically extracted from each print brochure. Reading grade level and reading ease scores were determined using the Flesch-Kincaid analysis, obtained via computer calculation in Microsoft Word, 2003.

Two reviewers (F.H-B., A.S.) evaluated materials for each of the criteria. Operational definitions used for objective evaluation of each criterion are listed in supplemental Table 3 (available in an online appendix at http://dx.doi.org/10.2337/dc07-1365). Final ratings resulted from discussion and consensus. Because of the document review procedures and operational definitions utilized, there was rarely disagreement between reviewers regarding whether a criterion was met.

Evaluation criteria are listed in Table 1. ADA materials successfully met 11 of 32 criteria (34%), and AHA materials met eight (25%). Overall, 6 of 23 literacy-demand criteria, including use of typographic cues (e.g., bullets) to emphasize key points, graphics design and layout, and font style and case, were achieved across brochures. Two of nine behavioral activation criteria were achieved by all brochures: inclusion of “how to” information and positive depiction of cultural images.

Readability criteria were least consistently met (Table 2). Overall, ADA brochures achieved lower reading grade levels; five brochures had an overall reading grade level that met the consensus recommendation of ≤5th grade. Wide ranges of required literacy were found within all brochures, however, with sentences reaching grade levels as high as 23.5 for ADA materials and 24.6 for AHA materials (surpassing postdoctoral education), impacting consistency of readability. Font size reduced ease of reading and vision accessibility. Use of multisyllabic or less common words (e.g., quivering, prone, vitally, overexertion, predisposes, gorge, and translucent), technical terms that were often not sufficiently explained (e.g., the explanation for PCBs was polychlorinated biphenyls), and inclusion of significant amounts of excess information nonessential to the purpose of the brochure also contributed to poor reading ease in brochures with low readability.

Criteria least often met were devotion of ≥50% of content to recommended behaviors and provision of step-by-step instructions to enable readers to enact recommended behaviors. When interactive activities were included, such as checklists and fill-in blanks to keep a record, small size of the activity logs/records often limited ease or practicality of use. Assessment of audience relevance and appropriateness criteria identified factors that might limit generalizability to low–health-literate and underserved audiences. Underlying assumptions were observed with regard to nutrition knowledge (e.g., “choose fats with two g or less saturated fat per tablespoon,” “be moderate in use of refined foods containing sugar”), availability of recommended foods (e.g., “keep fresh, low-fat foods around,” “enjoy fat-free and low-fat cheeses and other dairy products,” and “buy fresh fruits and vegetables”), access to specialty products (e.g., cholesterol-lowering margarine or “butter-flavored granules” as a butter/margarine substitute), and financial resources (e.g., recommendations to use choice cuts of meat and use a food processor). With regard to physical activity recommendations, underlying assumptions included existing exercise knowledge (e.g., “do 10 to 15 repetitions without resistance,” and the explanation that exercise will “maintain basal metabolic rate”), access to specialty recreation facilities (e.g., tennis courts, swimming pool for lap swimming, and hiking locations), and resource availability (e.g., models depicted wearing specialty exercise gear or sporting wear). These observable characteristics may not be perceived by underserved patients as representative of their experiences, environments, or resources.

Although the evaluated materials may be suitable for patients with adequate to proficient literacy and health literacy, there remains room for improvement in design of materials to meet the needs of patients with low literacy; low health literacy; vision difficulties; low prior medical, nutrition, and exercise knowledge; limited access to resources for fresh or specialty health foods; and limited access to resources for structured sporting and exercise facilities and activities. Unfortunately, these factors characterize underserved populations who have high rates of the diseases addressed in the education materials (9).

Concerted efforts to maximize the reach and effectiveness of print education materials, using available criteria, are warranted. Individuals with lower health literacy identify brochures or books as a primary source from which they receive health information (5), and print materials have proven effective as an intervention tool (10–12). Moreover, although auditory presentation of information is often recommended as a nonprint strategy to use with low-literacy patients, drawbacks include the associations of lower education and lower literacy with reduced auditory information processing and verbal memory (13,14). Supplementing discussion with printed information may improve understanding of health instructions (15) and processing of information for medical decision making (16).

A potential limitation is that the materials evaluated in the current study were obtained from the ADA and AHA Web sites, and patients with low health literacy may be less able or less likely to access these Web sites. Although the study procedure used the Web sites to ensure that all titles were identified and perused by the investigators, it is important to note that the ADA materials are designed as a “toolkit” for healthcare professionals, with specific instructions on the Web site for professionals to print out the brochures to give to their patients. Similarly, AHA materials are hard copy brochures that are made available in clinical and community settings, where patients of all literacy levels encounter them. A second limitation is that several of the AHA titles found on the Web site and requested from AHA were not received. Although these primarily reflected discontinued materials and changes to titles produced that had not yet been updated on the Web site, it is possible that some materials may have been unavailable as a result of such frequent request that they become out of stock. To the extent that we were not able to access some materials that may be used currently, it is possible that some titles are not represented in the current evaluation.

Development of such materials in the future would benefit from inclusion of experts in medical content, behavior change, education and literacy, and graphics/design. Moreover, patient feedback and empirical evaluation of accessibility, effectiveness, and most important criteria for optimal usability are essential (3).

This research was supported by grants from the National Institutes of Health/National Heart, Lung, and Blood Institute (K01 HL076644 to F.H.B.) and the American Diabetes Association (7-06-IN-07 to F.H.B.).

Parts of this study were orally presented at the American Heart Association Annual Scientific Sessions, Chicago, IL, 14 November 2006.