A Guide for Selection of Genetic Instruments in Mendelian Randomization Studies of Type 2 Diabetes and HbA_1c: Toward an Integrated Approach Free

Here we highlight recent examples from the MR literature where HbA_1c and/or diabetes genetic variants were used in MR studies, in what we are naming “an integrated approach.” An integrated approach to genetic instrument selection is one that considers factors that are sometimes overlooked in MR studies of glycemic traits. These include the use of novel approaches, such as, for example, that of Burgess et al. (57) described here; more careful consideration of which exposure GWAS is used; where possible prioritizing instrumentation of a continuous rather than a binary exposure; and, finally, ensuring that both the variance explained (R²) and measures of instrument strength (F statistic) are always calculated and presented.

In a recent MR study, Burgess et al. (57) used HbA_1c genetic variants to investigate associations between genetically instrumented glycemic status and incident coronary heart disease. The authors used a novel approach to genetic instrument selection: they took 40 independent HbA_1c, single-nucleotide polymorphisms (SNPs) based on their associations with diabetes at genome-wide significance from a recent GWAS (64) and their association with HbA_1c in the 2017 Meta-Analyses of Glucose and Insulin-related traits Consortium (MAGIC) GWAS by Wheeler et al. (61). They then calculated a weighted allele score for each individual in their data (UK Biobank [UKB]) whereby they multiplied each diabetes risk–increasing allele dosage by the SNP’s HbA_1c β-coefficient from the MAGIC GWAS. By doing so, the authors ensured that their allele score reflected average blood glucose levels, as opposed to only HbA_1c or risk of diabetes. This also relates to our earlier point about selecting instruments based on biological function. Corresponding metrics for their instrument were F = 144.5 and R² = 0.018, indicating that although they had fewer variants, this was a strong instrument, both in terms of total (R²) and average (F statistic) strength, and thus carried a low risk of weak instrument bias.

As mentioned earlier, an assumption that is often made in approaching genetic instrument selection in MR studies is that “bigger is better.” Therefore, researchers are likely to take as many SNPs (genome-wide significant and independent) as possible from the largest and latest GWAS. However, our own recently published MR study shows that this is not necessarily the case (69). We instrumented diabetes using both a 157- and 77-SNP genetic instrument, as we needed to try to mitigate issues of sample overlap between the GWAS for the exposure and the data under study (both UKB). Therefore, we took the 157 diabetes SNPs included in our instrument and looked them up in an older diabetes GWAS from 2014 (70). We found 77 of the diabetes SNPs (reduced number could be due to differences in coverage of imputation panels, for example) and observed that although this was an older GWAS in a different and smaller sample, the log(β values) for each SNP were comparable, even though most of the variants did not reach conventional genome-wide significance (P < 5 * 10⁻⁸). When we calculated the average strength (F statistic) of our 77-SNP instrument and compared this with the 157-SNP F statistic, they were 31 and 27, respectively. This indicates that an instrument with more genetic variants is not necessarily better in terms of average strength and greater the number of variants, i.e., the greater the likelihood of including pleiotropic variants.

A greater number of SNPs not always being better is also supported by recent MR studies that have instrumented BMI (71). The authors used an “older” instrument containing 96 BMI SNPs that performed well: Therefore, it is perhaps unnecessary to always use an instrument with hundreds of SNPs. Larsson et al. (71) showed that this BMI instrument explained 1.6% of the variance in BMI and had an F statistic of 61, while another recent MR study that instrumented BMI, for understanding of its association with chronic kidney disease, used a 773-SNP instrument, which explained ∼6% of the variance in BMI with only an F statistic of 23.6 (72). It is important to note that we need to balance these metrics against one another when selecting a genetic instrument for an MR study. This is because an instrument with more genetic variants has a larger R² (total strength) and more power but is also more likely to include pleiotropic variants, which could lead to violation of a core MR assumption. An instrument with a larger R² usually has a lower F statistic (average strength), which, if <10, will carry a greater risk of weak instrument bias.

We were interested in how many studies have instrumented HbA_1c and type 2 diabetes to date, whether there is a preference for one over the other, and whether they report metrics of instrument strength. Thus, we conducted a literature review in PubMed up until March 2021 (for details of our search terms and strategy see Supplementary Material) of MR studies that instrumented these exposures. We excluded anything that was not a research article, i.e., conference abstracts, letters, editorials, reviews, opinion pieces, and commentaries. Studies that evidently did not instrument HbA_1c or type 2 diabetes were not included. Supplementary Tables 1 and 2 list all the studies for diabetes and HbA_1c, respectively, that were included.

The aim of these empirical examples was to show the reader that 1) calculating R² and F statistic metrics as part of an MR study is important to understand both the total and average strength of the instrument of choice; and 2) irrespective of whether individual- or summary-level data are used for an MR study, options for obtaining these metrics are available. We chose two approaches, as there has not been quantitative comparisons of how they perform for glycemic instruments when considering both the R² and F statistic. These methods are “individual-level data regression” and Cragg-Donald F statistic.

The UKB is a cohort of ∼500,000 adults recruited across the U.K.’s general population, aged 40–69 years at baseline (2006–2010), for which more details are published elsewhere (73). For the empirical examples in the individual-level data regression and the Cragg-Donald method we used individual-level data from 349,326 UKB participants of White European ancestry, who had complete genotype (quality-controlled) and phenotype (type 2 diabetes and HbA_1c) data. Details of the genotype quality control can be found in our previous MR article (69). The UKB received ethics approval from the North West Multicentre Research Ethics Committee, and informed consent was obtained from participants.

For both phenotypes, we used previously described genetic instruments (69). Briefly, for type 2 diabetes the genetic instrument comprised of 157 SNPs from a 2018 GWAS of European ancestry (62), while the 51-SNP HbA_1c instrument came from a 2017 trans-ethnic GWAS (61). We filtered SNPs on minor allele frequency (>0.01) and used linkage disequilibrium (LD) clumping in PLINK, with P < 5 * 10⁻⁸ (69). For HbA_1c we also partitioned the instrument into 16 glycemic SNPs and 19 erythrocytic SNPs (the remainder are unclassified, as per the 2017 GWAS) separately with the aim of testing whether the HbA_1c instrument is strong in terms of both average (measured by the F statistic) and total (measured by the R²) strength when using all the SNPs, as well as with partition by biological function. Similar to our previously published MR study of glycemia and brain health/cognition/dementia outcomes, we suggest that it is worth doing three things when using an HbA_1c genetic instrument: 1) perform MR using all of the HbA_1c SNPs; 2) perform MR using only the glycemic SNPs; and 3) perform MR using only the erythrocytic SNPs.

This approach involves fitting a multivariable linear regression between SNPs and the exposure (treated as an outcome y here), where one evaluates the relationship between the j-th SNP and the outcome y while holding all the other SNPs constant. In the regression equation below, β₀ represents the constant and ε the residual or error term. As with any multivariable regression, the output includes the F statistic and R², which conventionally indicate the model fit, and in this case, we are likely not concerned with the interpretation of the coefficients of each SNP on the exposure. Linear regression can also be used when the exposure is binary (e.g., in this case, we used it for genetic liability for diabetes), whereby the coefficients and statistics represent associations on an absolute scale rather than a relative risk or odds ratio scale. Therefore, here we calculated R² and the F statistic for liability for diabetes using linear regression.

For this method we use the Cragg-Donald F statistic formula provided in the article by Burgess et al. (68), which requires a value for R² (previously calculated R² values were 0.028 and 0.030 for HbA_1c and 0.015 for diabetes), k (number of SNPs = 51, 275, and 157) and n (349,326). For consistency and comparability, we kept the R², k, and n the same as in the individual-level data regression approach above. Above, we were able to calculate the R², but it is sometimes the case that authors for GWAS provide the R² for the top SNPs, which could then be used in this formula.

Our search yielded a total of 657 studies for diabetes, of which 609 did not instrument this phenotype; thus, 48 remained. For HbA_1c, we found a total of 77 articles, of which 55 did not instrument HbA_1c and were excluded, leaving 22 articles. From this literature search it was clear that many more studies currently choose to instrument type 2 diabetes over HbA_1c.

As per Table 1 below, when using 51 and 275 HbA_1c SNPs in UKB, the individual-level data regression and Cragg-Donald formulae gave similar F statistics (using the same R² values of 2.8% and 3%). The two methods yielded somewhat different F statistics for the 16-SNP glycemic instrument, but both were substantially greater than 10, indicating no cause for concern (Table 1). For the 19-SNP erythrocytic instrument the F statistics obtained using both methods were comparable (Table 1).

Table 1 presents F statistics and R² metrics using both methods. Results were comparable irrespective of which formula was used (with the same R² of 1.5%).

The individual-level data regression approach naturally requires individual-level data for the exposure of interest, which are not always available to researchers. The Cragg-Donald formula, however, relies on having information about the R², which could come from the published GWAS for the exposure, yet this is not always included in GWAS papers. The “t statistic” approach can be used to calculate the F statistic when the R² is not known if β values or log(β values) and SEs are provided in the summary-level GWAS exposure data set. Thus, if individual-level data are available then the individual-level data regression may be recommended, but if this is not the case then the Cragg-Donald formula can be used.

Across our empirical examples in the UKB, the HbA_1c instrument outperformed the instrument for type 2 diabetes, in terms of total strength (R²) and average strength (F statistic), even though it contained markedly fewer SNPs. Specifically, the 16-SNP glycemic instrument had the highest average strength and explained 1% of the variance in HbA_1c, which is lower than the 2.8% variance explained for the 51-SNP instrument but certainly still appropriate for use in MR. The type 2 diabetes 157-SNP instrument had a much smaller F statistic (F < 30) for UKB overall and explained ∼1.5% of the variance in diabetes in UKB. On the other hand, the HbA_1c erythrocytic instrument also demonstrated that it is more than adequate for use in MR studies, with a similar R² to the glycemic variants and an F value of just under 200. Therefore, whether or not it is partitioned into glycemic and erythrocytic, the HbA_1c genetic instrument with 51 SNPs is overall a strong instrument for use in MR studies, as indicated by both R² and F statistic metrics, even in comparison with the newer 275-SNP HbA_1c instrument. However, the type 2 diabetes instrument appears to be somewhat weaker both in terms of total and average strength compared with the HbA_1c genetic instrument(s).

First, as demonstrated in our empirical examples and argued above, “bigger is not always better” when it comes to selection of instruments for glycemic MR studies. Above we show that in some cases glycemic instruments with fewer SNPs may be stronger and therefore more robust for use in MR when it comes to trying to minimize the important issue of “weak instrument bias.” This is the case for both HbA_1c and diabetes, with the HbA_1c instrument being superior. We therefore recommend that researchers do not assume that the latest and largest GWAS will always yield the best genetic instrument for these exposures and that careful consideration be given to which GWAS is selected for the exposure. Genetic variants identified in older GWAS may of course also be pleiotropic. Thus, researchers might choose to empirically test this in their MR study by, for example, performing a phenome-wide association study. However, it is important to note that in instrument selection one will likely have to balance choosing an instrument with a larger number of genetic variants (greater R² = total strength) and an instrument with lesser average strength (lower F statistic). When prioritizing the former, it is more likely that the instrument will include pleiotropic variants, which violates a core MR assumption. If the latter is prioritized, it is possible that the total instrument strength may be weakened, as fewer variants often yield a larger F statistic, but with lower variance explained in the exposure (R²). However, it is also important to note that more variants provide opportunities to use more robust methods, including common sensitivity analyses such as the MR-Egger test. However, for the HbA_1c instrument exemplified above in the UKB cohort, when we partitioned by glycemic versus erythrocytic variants, the R² remained at 1% for a small number of SNPs. Therefore, this example is a demonstration of an integrated approach with consideration of the total and average strength of the instrument, alongside biological function of the variants. In addition, another way to avoid pleiotropy is to use an approach such as that of Luo et al. (74), who adjusted for erythrocytic properties to control for unknown sources of pleiotropy.

Second, to reiterate the recommendation made by Boef et al. (75), and the more recent STROBE-MR guidelines (66), authors of MR studies should calculate and report the F statistic for the association between their genetic instrument and the exposure of interest in their study. As demonstrated earlier, this can be calculated using one of three approaches, depending on whether researchers have access to individual-level data. If individual-level data are available for the exposure of interest, then researchers should likely prioritize calculating the F statistic using the individual-level data regression approach. If individual-level data are not accessible, but the exposure GWAS article provides the R² for the (exact) instrument that is being used, then we recommend using the Cragg-Donald F statistic method. An additional method exists, namely the t statistic method, which we did not implement here. This is because the t statistic method (F = β²/SE²) can be used when the R² is not known (i.e., not provided in the article for the GWAS for the exposure). In this equation, β represents the coefficient for each SNP’s association with the exposure and SE its standard error. By using the t statistic method, the obtained F statistic will be more of an approximation because the discovery GWAS (usually for the exposure) sample size is used rather than that of the outcome data set.

Third, and related to our earlier point, there are some complex issues surrounding genetic instrumentation of binary disease exposures such as diabetes (76). In instrumenting these types of disease exposures, it is important to note that we are modeling an underlying continuous measure where liability thresholds are used to separate individuals into different categories (76,77); thus, we should interpret MR using binary exposures in terms of genetic liability (77). If MR instrumental variable assumptions are met for the underlying continuous exposure that is used to categorize individuals, then we assume that we can infer causality using the binary exposure (76). However, there may be circumstances in which researchers feel the need to genetically instrument diabetes itself as it may prove to be clinically informative. We still recommend that researchers interested in how hyperglycemia might causally impact a range of important health outcomes take advantage of what is evidently a strong HbA_1c instrument. This instrument is currently underused, as we found only 22 studies where it was used as an exposure in MR studies, and thus we recommend that researchers exploit this instrument to a much greater extent. Also, the MAGIC GWAS do not include UKB, making this instrument very attractive for use in two-sample MR studies of HbA_1c and important health outcomes. In terms of instrument metrics, our applied example in the UKB data clearly showed that the HbA_1c instrument completely outperformed the diabetes instrument. The HbA_1c instrument can also be split by biological function, into erythrocytic and glycemic SNPs, as shown above in our examples. Genetic instrumentation of a continuous exposure, such as HbA_1c, also enables the application of nonlinear MR methods (78), which are also somewhat underused in MR. Using nonlinear MR methods can help define levels of risk and may also aid in understanding that it is both low and high levels of HbA_1c that are associated with risk. While understanding the causal impact of disease status (e.g., diabetes) on a range of outcomes is both interesting and important, it is well established that continuous measures are superior and should be used where possible.

Fourth, we recommend that where plausible, researchers adopt an instrument selection approach such as that of Burgess et al. (57), which we described above (example A) with the aim of illustrating a novel line of thinking to integrate both diabetes and HbA_1c into an MR study. In this study authors used a method that exploited properties of each of these exposures, and this yielded an instrument with good average (F = 144) and total strength (2.8% variance explained). An alternative form of biological integration is illustrated in the works of Yuan et al. (79) and Au Yeung et al. (80) who integrated expression of relevant genes and HbA_1c in their instrument selection process.

Fifth, another example of an integrated approach to instrument selection is provided in example B above, in which we sought to bypass the issue of sample overlap in our previous MR study. To try to mitigate this we took as many of the newer diabetes variants as possible (from a more recent GWAS that contained overlap with our data under study) and used the effect estimates from the earlier GWAS. The most popular approach to instrument selection is, naturally, to take the most recent, largest GWAS (which often includes the UKB), due to assumptions that the benefits (e.g., large number of genetic variants) outweigh the risks (e.g., sample overlap). However, we show that a diabetes instrument with 77 SNPs had a larger F statistic (average strength), indicating that, if anything, this instrument carried a lower risk of weak instrument bias compared with our original 157-SNP instrument.

While our article focuses on genetic instrument selection for MR studies of HbA_1c and/or liability for diabetes, we acknowledge that as a method, MR has limitations and is not a panacea for causality. As such, triangulation of findings is crucial whereby different study designs are used to enable robust causal statements. Key limitations of MR include confounding by ancestry, confounding by LD, confounding by horizontal pleiotropy, and canalization (81). Confounding by ancestry, or population stratification, refers to the fact that allele frequencies of common genetic variants, as well as disease frequencies, may differ by population. However, it is now common to adjust for genetic principal components in MR studies to correct for residual confounding by population structure. Confounding by LD refers to when the selected genetic variant(s) is/are in LD with (i.e., correlated with) another genetic variant associated with the outcome under study, which may produce a confounded causal estimate. Confounding by horizontal pleiotropy is when a single genetic variant influences the outcome under study directly rather than via the exposure being instrumented. However, numerous methods have been developed to detect and correct for horizontal pleiotropy (82). Canalization is when an individual develops a compensatory mechanism for disruptive genetic or environmental influences, as a response to higher or lower levels of a risk factor (e.g., higher or lower BMI).

In summary, we recommend that MR studies of glycemia take a more integrated approach when it comes to selection of genetic instruments. Therefore, careful consideration should be given to the following: 1) whether novel approaches such as those described here from the literature might be used; 2) which GWAS is used to select the instrument for the exposure; 3) whether a continuous, as opposed to a binary, exposure can be instrumented; and 4) inclusion of both variance explained (R² = total strength of the instrument) and the F statistic (average strength).

Acknowledgments. The authors thank the volunteer participants of the UKB and the UKB researchers. The authors acknowledge Dr. Chloe Park (MRC Unit for Lifelong Health and Ageing, University College London) for her help with creating the graphical overview.

Funding. This work was conducted under UKB project no. 7661. V.G. is jointly funded by British Heart Foundation (SP/16/6/32726) and Diabetes UK (15/0005250) and is supported by the Professor David Matthews Non-Clinical Fellowship from the Diabetes Research and Wellness Foundation (SCA/01/NCF/22). A.S. and N.C. are supported by the U.K. Medical Research Council (MC_ST_LHA_2019, MC_UU_0019/2). S.B. is supported by a Sir Henry Dale Fellowship jointly funded by the Wellcome Trust and the Royal Society (204623/Z/16/Z). This research was supported by the National Institute for Health Research Cambridge Biomedical Research Centre (BRC-1215-20014).

The views expressed are those of the authors and not necessarily those of the National Institute for Health Research or the Department of Health and Social Care.

Duality of Interest. N.C. receives funds for serving on clinical trial data safety and monitoring committees sponsored by AstraZeneca. No other potential conflicts of interest relevant to this article were reported.

Author Contributions. V.G. conceived the idea for the study and performed the analyses. V.G. and A.S. each conducted the literature search for MR studies of diabetes/HbA_1c. S.B. provided important intellectual input. N.C. supervised the project and provided important intellectual input. All authors provided feedback on drafts of the manuscript and approved the final version.

Data and Resource Availability. The UKB data are available at www.ukbiobank.ac.uk/using-the-resource/. This study was conducted using the UKB resource, application identifier 7661.