In this issue, Norris, Engelgau, and Narayan (1) provide a systematic comprehensive examination of 72 randomized controlled trials (RCTs) published since 1980 that tested various forms of self-management training (SMT) in type 2 diabetes. By their definition, SMT includes any instruction or counseling involving lifestyle behaviors (such as diet, physical activity, skill development, and coping), as well as the provision of knowledge/information. Their conclusion that “evidence supports the effectiveness of self-management training in type 2 diabetes, particularly in the short term,” is not as strong or prescriptive as diabetes educators, educational researchers, and policy makers would like it to be, given the seemingly large number of “gold standard” RCTs. Indeed, except for the clear call for further, more focused, and better research, strong evidence-based conclusions and recommendations about diabetes education remain elusive.
Virtually all diabetes care providers intuitively recognize that diabetes education/SMT is an essential component of diabetes treatment. Why aren’t more studies published that provide strong evidence-based guidelines for diabetes education?
One possible answer is that few studies have been conducted; however, this is not the case. Every year Diabetes Care receives a substantial number of descriptive as well as analytical educational research manuscripts. Unfortunately for the authors, many are rejected outright because of methodological issues, e.g., seriously flawed study design. Others are subject to major revisions for both fixable problems, such as inappropriate statistics or conclusions not reflecting results, and those not fixable after study completion, such as lack of appropriate comparison groups. Quality educational research is imperative as policy makers increasingly use evidence-based data to develop clinical care and reimbursement guidelines, as health professional education becomes increasingly evidence-based, and as voluntary health organizations such as the American Diabetes Association base position statements on evidence-based studies. For these reasons, Diabetes Care will continue as a leading venue for publishing only diabetes educational studies that meet or exceed standards for quality research set by its editors, reviewers, and critical readers.
That having been said, as Norris, Engelgau, and Narayan (1) point out, even the best studies often have flaws, limiting their power to define the best practices. After all, educational research is subject to a host of design complexities, making it very difficult, if not impossible, to carry out a perfect study. Whereas sharing knowledge/information may seem straightforward, providing other SMT components (such as helping people to understand the consequences of their choices, giving them the skills they needed when they elect to make healthy choices, and helping them to learn problem-solving strategies for coping with challenging situations) is not straightforward. Working with free-living individuals, educational researchers simply have less control over the conditions under which interventions take effect than is generally possible with pharmacological clinical studies, metabolic unit studies, or animal studies.
Many of the studies reviewed by Norris, Engelgau, and Narayan (1) were subject to confounding variables (e.g., medication changes when education interventions were not integrated with medical care) that may have affected biochemical outcomes. In addition, because many investigators think it is unethical to have a noneducated control group, most controlled educational studies compare a special, generally more intensive intervention to basic care and education (usually not defined). Such basic care is sometimes quite substantial, minimizing the apparent effects of the special intervention. On the other hand, control-condition participants generally receive less attention. In such cases, intervention group outcomes may arguably be due to attention factors (the well-known Hawthorne effect). Large RCTs, such as the Diabetes Prevention Program (2), can directly compare lifestyle to medication intervention, eliminating the potential medication bias; however, the level of attention given to members of the different groups varies.
Strong evidence-based guidelines for diabetes education are still lacking, partly because educational research is fraught with threats to internal validity (selection, performance, attrition, and detection bias). In fact, in diabetes educational studies, it is impossible to blind patients to the intervention, and thus performance bias is always present. Norris, Engelgau, and Narayan (1) used Cochrane Library (3,4) methodology for evaluating internal validity and found that not 1 of the 72 published studies reviewed was free of bias (excluding performance bias). Attrition bias is prevalent among education research studies submitted to Diabetes Care. High dropout rates before study completion represent a potential loss of control of the study, particularly if dropouts are not compared with those completing the study or if they are found to not be equivalent at baseline to those completing the study. Whereas using a rigorous run-in protocol can reduce a dropout rate, it may cause study subjects to be less similar to the population at large. Despite the overwhelming success of the Diabetes Control and Complications Trial (DCCT) in proving that metabolic control is important (5), translating the DCCT findings to clinical practice has proven difficult, partially because of selection bias.
We salute Norris, Engelgau, and Narayan (1) for avoiding the temptation to combine unlike studies using meta-analytic statistical techniques. The educational interventions described in the 72 studies varied dramatically from one another on many levels. Unfortunately, however, the reports often failed to include essential information needed to understand and to replicate the interventions. Not all diabetes education is equal, and one aim of diabetes education research must be to identify which conditions and factors are important for which groups or types of patients. Therefore, participants and the populations from which they are drawn must be described carefully. In addition, we call on authors of patient education research to report, at a minimum, several descriptive and theory-based characteristics of their treatment and control group interventions, and we encourage reviewers and editors to insist that participant and population characteristics be well defined. These characteristics include the following: site (e.g., patient’s home, inpatient, or outpatient), format(s) (e.g. one-on-one, family, or group), mode of delivery (e.g., face-to-face, telecommunication, or computer), teaching method(s) (e.g., didactic, goal-setting, or situational problem-solving), content(s) (e.g., nutrition, exercise, self-monitoring of blood glucose, or medications), and provider (e.g., nurse, dietitian, physician, psychologist, or person with diabetes). Authors should also report whether the intervention: 1) was individually tailored to participants, based on an initial assessment; 2) included follow-up assessment and modification; 3) was preceded by a baseline education supplement, also given to comparison group members; and 4) was coordinated with the provider(s) of medical care. Finally, the intensity of interventions should at least be characterized in terms of the number of educational encounters, the duration of each episode, and the span of time over which the program was provided (6). Without this information, it is virtually impossible for other researchers to duplicate the study or for clinicians to replicate successful interventions.
We are not suggesting that although quality educational research is imperative, it cannot be done well. Despite significant challenges, successful and significant educational studies have been and will be completed. To improve the quality, publishability, and the meaningfulness of educational research, we call for the following:
Greater collaboration among educators, care providers, research design experts, and statisticians. Initial input from a range of specialists will help address important methodological and design pitfalls.
Increased focus and definition of the issues being studied. When resources are limited, as they usually are in educational research, compromises in design, subject recruitment, interventions, and assessments must inevitably be made. As with biomedical research, educational research must be as unbiased as possible, hypothesis-driven, designed to evaluate testable hypotheses, and focused on important measurable end points. It requires discipline to know when an educational study is trying to accomplish too much and is therefore doomed to fail in accomplishing the primary aim.
Conscientious realistic expectations of reviewers and consumers of educational research. The overall evaluation criterion applied to a patient education research study should focus on its potential to further understanding in the field. The editors of Diabetes Care and its reviewers often struggle with manuscripts describing program evaluations that fall short of the RTC. Nevertheless, if these reports contribute new and important findings, the best are published in hopes that subsequent, better-controlled studies will build on them.
In the final analysis, planning, conducting, and publishing quality diabetes educational research is not simply a matter of knowledge and willpower, but one of sufficient resources, including greater government and voluntary agency funding designated for RTCs. The challenge for researchers is to fit all of these necessary pieces together into a workable whole. If this is accomplished, we should expect to have many more than 72 quality RTCs published in the next 20 years.
We thank Dave Marrero, PhD., Indiana University School of Medicine, and Tom Elasy, MD, MPH, Vanderbilt University School of Medicine, for reviews and comments.
Address correspondence to Madelyn L. Wheeler, MS, RD, FADA, CDE, Coordinator, Research Dietetics, Diabetes Research and Training Center, IF 122, 250 University Blvd., Indianapolis, IN 46202. E-mail: firstname.lastname@example.org.