Core Outcome Sets for Studies of Diabetes in Pregnancy: A Review

Worldwide estimates suggest that hyperglycemia in pregnancy affects 21 million live births annually (1). Approximately 85% of these are due to gestational diabetes mellitus (GDM) with the remainder a result of preexisting maternal diabetes (2). Diabetes poses an increased risk of multiple adverse outcomes for both mother and baby during and after pregnancy. In 1989, the St Vincent Declaration set a 5-year target for approximating outcomes of pregnancies in women with diabetes to those of the background population (3). While these goals have not been reached, the past 30 years have witnessed an intensive research effort to improve outcomes for women with diabetes and their children.

When designing a clinical trial, and indeed any research study, researchers must decide which outcomes to measure and report. Multiple factors influence this decision including responsiveness of the outcome to the intervention, cost of measuring an outcome, and acceptability and importance of an outcome to study participants (4). Outcomes reported from diabetes in pregnancy studies differ widely, and this lack of consistency makes meaningful comparisons between studies difficult (5). It also limits our understanding of intervention effects. For example, a systematic review and meta-analysis of 21 studies of prepregnancy care (PPC) for women with pregestational diabetes noted that just 13 studies included congenital malformations as an outcome and only 5 included changes in the level of glycated hemoglobin (HbA_1c) (6). Reporting bias is an additional concern within scientific literature as a whole. Studies with positive results are more likely to be published, and a review of cohort studies assessing outcome reporting bias in randomized controlled trials demonstrated that 40–62% of studies had at least one primary outcome changed, introduced, or omitted (7). Finally, there is concern that selected outcomes do not always reflect the values and preferences of study participants, who are key stakeholders in development and assessment of clinical interventions. For example, a study on dietary advice for GDM management found that stress and anxiety were reported as important outcomes from the woman’s perspective. Still, these are reported rarely in existing GDM-management literature (8).

In an attempt to overcome these issues, there is a move to develop core outcome sets (COS) in the field of diabetes and pregnancy and other areas of research and clinical practice. A COS is the minimum set of outcomes or outcome measures and is a consensus-driven recommendation of “what” should be measured and reported in all studies for a given health issue (4). In addition, there is the recognition that COS are also important in informing decisions about outcomes to be recorded in routine clinical data and for clinical audit/quality improvement projects. Of course, the aim of a COS is not to curb innovation, and so researchers are free to measure and report additional outcomes of interest. Due to sample size and cost limitations, it is not expected that individual studies are powered to examine differences in all specified outcomes within a COS; however, measurement of the components will facilitate future combining and comparing of multiple studies within a field of research. The Core Outcome Measures in Effectiveness Trials (COMET) initiative brings together people interested in the development and application of such outcome sets. It has developed a handbook and additional reference material that provide detailed guidance on COS development (4,9).

The concept of standardizing outcomes is not novel. In the 1970s, the World Health Organization led a collaboration resulting in the World Health Organization handbook of guidelines recommending the minimum requirements for data collection in cancer trials (10). More recently, the Outcome Measures in Rheumatology (OMERACT) collaboration has driven the development of COS in rheumatoid arthritis and other rheumatic diseases (11). To streamline COS development, the Core Outcome Set–STAndards for Development (COS-STAD) (12) provides minimum standards to be followed by COS developers and the Core Outcome Set–STAndardized Protocol items (COS-STAP) consists of a checklist of 13 items considered essential documentation in a COS protocol (13). The Core Outcome Set–STAndards for Reporting (COS-STAR) statement is a helpful resource to standardize COS reporting (14).

This review summarizes completed and ongoing COS development in the area of diabetes in pregnancy, reviews the role of patient and public involvement (PPI) in COS development, and discusses opportunities for future progress in this area.

The PubMed database and COMET registry (www.comet-initiative.org) were searched for English-language studies and COS publications. The following search terms were used alone and in combination: “diabetes,” “pregnancy,” “COS,” and “core outcome set.” A date restriction was not applied. Results were reviewed by the authors and selected for inclusion based on relevance to the topic. Additional articles were identified by manual searching of reference lists of included articles. The patient perspective was provided by coauthors (C.M. and C.O.) who have experience as patient and public representatives in COS development as part of the INternational collaboration for Studies in PREgnancy and Diabetes (INSPIRED) research group. The information was obtained from their written responses to open-ended questions reflecting their experience as COS developers.

In 2012, Bennett et al. (15) used a systematic review with stakeholder input to identify clinically important research questions and high-priority outcomes for the management of GDM. This review was one of the first attempts to address outcome reporting in this field. Acknowledging that waste in research may result from important outcomes not being measured or reported in clinical trials and reviews in Australia, the WOMen and Babies health and wellbeing: Action through Trials (WOMBAT) Collaboration recognized a critical need to standardize outcomes. Following review and consensus discussion, the group developed and disseminated a useful list of standardized outcomes for GDM. Bain, Middleton, and Crowther (16) examined the use of these GDM outcomes in Cochrane protocols and reviews before and after the publication of the WOMBAT outcomes list. The authors found an increase in the number of prespecified outcomes reported over time in Cochrane reviews attributed to the WOMBAT initiative. The authors emphasized a need to move further toward an international COS for GDM research. In 2014, recognizing that variation in outcome collection and reporting is a serious hindrance to progress in the specialty of women’s and newborn health, >80 journals came together to endorse the CoRe Outcomes in Women’s and Newborn health (CROWN) initiative (17). As a consortium, CROWN encourages the development of COS using robust consensus methodology and aims to organize robust peer-review and effective dissemination of manuscripts describing COS. It is hoped that this approach will facilitate the embedding of COS in research practice and encourage close collaboration between researchers, reviewers, funders, and guideline makers.

Following this call for action, three COS have been published in the field of diabetes in pregnancy by the INSPIRED research group. This group comprises researchers, health care professionals, and patient representatives from a broad geographical base, including representatives from low-/middle-income countries, who work together in a committed way to improve research in this field. Recruitment to the research group has taken place by raising of awareness at international meetings and within special interest groups by e-mails to the chairpersons and individual members where possible. Thus far, the group has had an open-door policy to membership and welcomes new and interested stakeholders. Their published COS focus on studies evaluating the effectiveness of PPC for women with pregestational diabetes (5,18), follow-up at 1 year and beyond of women with GDM treated with insulin or oral glucose-lowering agents (19,20), and studies of GDM prevention and treatment (21,22). The INSPIRED research group is currently developing a COS for studies of treatments for women with pregestational diabetes, and a study protocol on the development of a COS for diabetes after pregnancy prevention interventions (COS-DAP) has been published by another international group of researchers (23,24). Table 1 outlines these completed and ongoing studies.

While there are a variety of approaches to COS development, the studies in Table 1 have followed a process involving three distinct steps (Fig. 1), as follows.

This generally involves a standard approach with multiple database searches and minimal time limits to generate a long list of outcomes. However, in specific disease areas, the task of examining all previous literature may be overwhelming and require intensive use of resources. One pragmatic approach to overcoming this while also capturing the majority of outcomes is to conduct the systematic review in stages defined by year of publication until outcome saturation is reached (4,21).

Delphi surveys are used frequently during this step. This iterative method was developed at the Rand Corporation in the 1950s and aims to achieve convergence of opinion in sequential questionnaires sent by post or electronically (4). Typically participants are asked to rate the importance of including each outcome in the COS on a Likert scale with an option to suggest additional outcomes that are not listed. Participants then have the opportunity to consider the views of other participants before rerating each item on subsequent surveys (typically two or three rounds in total). After the initial survey round, some items may be dropped according to prespecified criteria (21).

This is traditionally a face-to-face meeting where stakeholder representatives discuss the survey results and vote on each outcome for inclusion in the COS. Again, prespecified criteria should be used to structure the meeting and define consensus. An experienced facilitator is essential to ensure that all parties are included in the discussion and that there is adherence to the prespecified approach in achieving consensus (21). The feasibility and potential barriers to measuring each outcome should be discussed. This will improve the future implementation of the final COS.

Key to successful COS development is identifying and involving key stakeholders. Ideally, participants will represent all key stakeholders and commonly include health care practitioners, trialists, researchers, and policy makers. However, the expertise of patients and carers in development of COS is critical and well recognized (25). Indeed, the outcomes on which research studies report should be important and relevant to those who will potentially receive care based on the findings. The guidance offered by the COMET initiative and included in the COS-STAD recommendations highlights the contribution patients and their families/carers can make in COS development (4,14). This contribution typically presents as follows:

• 1) PPI as part of the research team involved in planning/designing and conducting the study and

• 2) Patient and public stakeholders in the study as research participants

The distinction between involvement and participation is not always explicit in published reports of COS (25); however, both warrant consideration in future studies including those within the context of diabetes in pregnancy.

PPI in the context of point 1 (noted above) relates to the inclusion of “public research partners” and is underpinned by the definition offered by INVOLVE (a U.K. national advisory group to support active public involvement in research) that research is carried out “with/by” rather than “for/about” members of the public (26). The COMET initiative suggests that PPI can facilitate this by their presence on the Study Advisory Group (4). Such involvement ensures that the voices of patients and the public contribute to the focus and design of the study, provide input at each stage of the research process, and are part of the dissemination strategy (25). PPI can, and should, bring insight into methods for the design, conduct, and reporting.

As the methodology of COS development has advanced, the inclusion of patients and the public as research participants has also increased (27). In its broadest sense, this is viewed as a means to ensure that outcomes important to patients are included in the COS (28). While the best approaches for facilitating patient inclusion are as yet unknown (25), guidance is offered by the COMET initiative and by participants and researchers of previous COS studies (25,28–30).

It is acknowledged that the systematic review approach typically used in the first stage of COS development could lend itself to identifying outcomes that are important to researchers only (25). Qualitative research, using data collection methods of individual and/or focus group interviews, can enhance this stage by identifying outcomes that are important to patients and the public and therefore include the wider stakeholder community (29). In-depth exploration using qualitative research techniques may help uncover outcomes engrained within the experiences of patients that might otherwise be missed. Qualitative approaches can also give insight into why some outcomes are considered more important than others to specific stakeholder groups (31). Delphi surveys have been identified as the most commonly used way to ensure patient and public participation in COS development (28). Published COS studies of diabetes in pregnancy have used this method whereby health care professionals, researchers, and women (with a history of diabetes) have participated across the Delphi rounds (18,20,21). Researchers are cautioned, not without its challenges, to ensure that outcomes are presented in a way that is accessible to all groups (e.g., plain-language descriptions are usually required for all groups and not just for stakeholders without a clinical background) (27). The final phase of COS studies involves a consensus meeting to agree on the final list of outcomes. The COMET initiative notes that some consensus meetings include all stakeholders, while others may run a separate meeting for the patient and public participants (4). The published COS for studies of diabetes in pregnancy facilitated all stakeholder groups, including representatives of women with diabetes, within one consensus meeting (18,20,21). Bringing all participants to the discussion gave perspective on what was held as important to all groups (20). Table 2 outlines key sentiments of two of our coauthors: women with diabetes who have experience in COS development and self-managing diabetes during pregnancy. Overall, they described a positive experience and recognized the value of their contributions. However, these women were selected based on their prior experience and involvement with COS development and may not represent all women with diabetes, particularly from a cultural perspective. Further exploration of the public and patient’s experiences of participating in COS development is warranted.

A working group (People and Patient Participation, Involvement and Engagement [PoPPIE]) established within the COMET initiative focuses specifically on the public’s involvement and participation in the development of COS. Future COS work, including studies of diabetes in pregnancy, should ensure that methods supporting involvement and participation are sustainable, meaningful, and evaluated robustly.

We also suggest that COS publications include a tailored PPI statement describing the PPI involvement including at what stage in the study the patients/public first became involved and how their concerns and preferences informed the developed outcome measures.

As COS development progresses, it is important to identify and prioritize areas within the field of diabetes and pregnancy requiring COS and explore how best to address this need. OMERACT has established working groups for subspecialty areas in rheumatology where members work virtually and meet at a biennial conference (11). There is not a similar working group for diabetes and pregnancy, but the James Lind Alliance Priority Setting Partnerships were established in 2019 to identify areas for research to improve the health and well-being of mothers, babies, and families affected by diabetes in pregnancy (32). They aim to produce a top 10 list of research questions that women, their support networks, and health care professionals agree are the most important for research to address in diabetes and pregnancy. There is potential for COS developers to establish a working relationship with the James Lind Alliance and bring together patients, carers, and clinicians.

Defining the scope of each COS is important and should be clarified during COS protocol development. When developing lists of previously reported outcomes using systematic review methodology, COS developers typically include randomized trials or systematic reviews of randomized trials (19,22). However, in areas where randomized trials are limited, the systematic review has included additional study designs such as prospective cohort studies and case-control trials (5). This raises the question of whether a COS is relevant to all study types in a specific area. For example, will a COS be relevant to basic science as well as clinically focused studies in a specific field? We would argue that, if available, an appropriate COS should be used for all forms of research. Ideally, COS developers will present a list of study types that might be within the purview of the COS at the time of publication; however, study designers will ultimately need to judge whether a published COS applies to their particular research question(s).

The global applicability of any particular COS is currently unclear. In particular, developing countries are often heavily underrepresented during COS development and the geographical distribution of participants in any specific COS is usually focused around certain regions (18). This may be related to lack of resources (both time and financial) needed to participate in the process or a perception that the final product will not be relevant to their research or clinical practice. Strategies such as translating surveys into different languages and having a facilitator engage with stakeholders during the Delphi process could be helpful but have not been evaluated. Relating to this point, international guidelines can recommend different approaches to diagnosis and management of a particular condition depending on available resources and infrastructure, even if not supported by high-quality evidence. This is the case for GDM and is based on the premise that some structured care is preferable to no care at all (33). With this in mind, it may be necessary to develop different COS (or COS modifications) for different clinical settings. While further guidance is awaited, there is an onus on COS developers to consider the practicality and cost when recommending an outcome for inclusion in a COS.

It is worth noting that COS studies are designed typically to identify “which” outcomes are important to measure—not necessarily “how” to measure these outcomes. The International Association of the Diabetes and Pregnancy Study Groups (IADPSG) Working Group on Outcome Definitions has already published a repository of definitions that is a handy resource for researchers once the core outcomes have been selected (34). This repository includes both maternal and fetal outcome definitions and was assembled using a thorough systematic review to identify previously reported definitions, followed by an expert review to ensure accuracy. Unfortunately, there is not yet an international consensus on the best approach to diagnose GDM, which is responsible for the majority of cases of diabetes in pregnancy.

Newer techniques such as real-time Delphi offer potential “roundless” approaches to achieving consensus but have yet to be explored comprehensively in COS development (35). This method encourages respondents to revisit an online questionnaire as often as they want within a specified time frame. Participants can see their responses as well as the updated answers of others. This has the potential to allow for dialogue between respondents and reach consensus in a more time-efficient manner. This automated approach could significantly reduce facilitator burden, but issues such as maintaining stakeholder engagement throughout the process need to be examined. It will be necessary to formally evaluate real-time Delphi against the more traditional approach to assess what, if any, differences exist.

Digital communication technologies have rapidly advanced in the past decade, and the coronavirus disease 2019 pandemic has witnessed a surge in their use both within and outside of the workplace. In the face of adversity, the medical community has adapted significantly to facilitate ongoing research, education, and patient care in this new era of social distancing and shelter-in-place orders. Our increased familiarity with teleconferencing platforms and etiquette can be used to broaden the geographical base and make it easier for stakeholders (including PPI) from a range of low- to high-income countries to actively participate in COS development. For example, it now seems reasonable to consider replacing the traditional face-to-face consensus meeting with an online gathering as is anticipated with an ongoing COS (24). This approach has the potential to save time and money, and sessions can be easily archived for future review. Removing the cost and time associated with traveling may allow greater participation from those in lower-income and geographically distant countries. Supported by an experienced facilitator and using electronic polling methods, a technology-enabled meeting may result in less peer pressure between participants and a more inclusive COS.

Another area of debate is how frequently a COS should be revised. Periodic review will allow developers to confirm the ongoing relevance of the COS by ensuring all outcomes are still important and that no additional outcomes should be added. Perhaps this periodic review could be the responsibility of a lead group who would perform this task on behalf of all diabetes in pregnancy stakeholders. For example, the International Headache Society Committee on Clinical Trials published guidelines for studies of pharmacological treatments for tension headaches in 1995 (36). They specified that a revised guideline was planned for 3–4 years from that date, and although additional time lapsed, an updated report was published in 2010 (37). The potential of a “living COS,” where the COS is updated continually as new, relevant evidence becomes available, should also be explored. This is already being done in the systematic review field whereby living systematic reviews are underpinned by continual monitoring of the evidence; the inclusion of new, relevant evidence; and a process for communicating the up-to-date status of the review. Outcomes in the COS will likely be more temporally stable than data informing living systematic reviews. Nevertheless, the processes for omitting already included outcomes and including new outcomes in a living COS require careful thought.

COS developers have an opportunity, and arguably a responsibility, to galvanize COS implementation and disseminate knowledge of the utility of COS and their potential value for future research. This has been achieved in other areas of health care where COS development is more established. For example, >80% of relevant clinical trials now use the rheumatoid arthritis COS (38). Prior work has solicited and evaluated dissemination ideas from COS research participants and listed strategies to improve the relevance, usefulness, and comparability of outcomes in clinical practice (39,40). Drawing on this information, we propose the following roadmap for COS implementation in the field of diabetes in pregnancy (Fig. 2).

Prospective online registration of a planned COS with COMET is an important first step for all COS developers (24). We recommend expansion of the currently available online COS resources to include formal training programs for researchers, health care professionals, and patient representatives. This will raise awareness of the role of COS and COS methodology and facilitate a quality framework around COS development.

During COS development, stakeholder representation should be as inclusive as possible from the time of study conception. Consideration should be given to inviting representatives from key scientific journals, funding bodies, special interest groups, societies, and national health departments. For example, although members of the INSPIRED research group are also members of key diabetes and pregnancy organizations, achieving formal endorsement would almost certainly increase engagement with COS development and downstream use of the final product.

Progress has clearly occurred in this regard with major funders such as the National Institute for Health Research in the U.K. and the Health Research Board in Ireland highlighting the importance of, and need for, COS to researchers seeking funding for new trials (4). In the U.S., the National Institutes of Health encourages the use of common data elements in clinical research, patient registries, and other human subject research with the aim of improving data quality and synthesis (41). A data element is information that describes a piece of data to be collected in a study, and so this concept is broadly similar to a COS, although COS are largely outcome-only focused. An increase in uptake of the rheumatoid arthritis COS was noted after its endorsement by the U.S. Food and Drug Administration in 1996 and the European Medicines Agency in 1998 (42). We propose that major funders take the additional step of insisting that grant reviewers ask authors whether a COS relevant to their study was available and whether it was used to identify and measure study outcomes. The response should be taken into account in judging of the submitted work. Similarly, while a large number of journals in the field of women’s health support COS development through the CROWN initiative, we suggest including information on COS in their “instructions for authors” web page and specifically asking authors during the submission process whether a relevant COS was used.

The approach to achieving widespread dissemination of a completed COS should be carefully considered. Along with traditional methods such as presenting at scientific conferences and publishing the COS in a high-impact journal, researchers should also consider the role of social media outlets and other online platforms in their approach. Indeed, electronic modes of dissemination including social media efforts and e-mails to stakeholders and professional groups are associated with an increase in COS stakeholder interest (39). COS developers can take the lead in informing research funders in the area of health or social care, guideline producers, and Cochrane review groups on completion of a relevant COS (4).

The uptake of an individual COS should be assessed regularly. With this strategy, the impact of specific dissemination activities can be evaluated and barriers to dissemination and uptake can be examined and addressed. This approach can also yield unexpected information requiring further exploration. For example, it has recently been noted that commercially funded trials are more likely to measure the rheumatoid arthritis COS outcomes compared with those without industry funding (39).

COS are a means to facilitate research in a particular area of health care that will address outcomes of importance to key stakeholders. Using a COS does not limit researchers from collecting additional outcomes of interest and relevance to their study. COS development has progressed in the field of diabetes in pregnancy and has the potential to reduce heterogeneity between trials and improve reporting. This should translate into improved evidence synthesis and knowledge transfer to reduce research waste, better inform clinical practice, and result in improved pregnancy outcomes for women with diabetes.

Duality of Interest. No potential conflicts of interest relevant to this article were reported.

Author Contributions. A.M.E. and D.B. conducted the literature review. D.B., O.K., L.B., P.M.O., D.D., and F.P.D. contributed to the manuscript writing. C.M. and C.O. have expertise in COS development as patient representatives with a history of pregestational diabetes. They provided their personal experience of PPI in COS development and contributed to the manuscript writing. All authors revised the manuscript critically for important intellectual content and approved the final version to be published. A.M.E. coordinated the drafting and circulating of the manuscript and is responsible for the integrity of the work as a whole.