7 Takeaways: reading MR critically

We have built the standard MR toolkit from scratch and watched it recover a causal effect we planted, then defend that effect against pleiotropy. The last and most useful skill is turning the method around: using what you now know to judge an MR analysis you did not run.

7.1 What we established

Method	Estimate	Verdict
Naive OLS	0.443	biased by confounding
2SLS (one-sample)	0.351	recovers truth
IVW (valid)	0.357	recovers truth
IVW (pleiotropic)	0.499	biased by pleiotropy
MR-Egger (pleiotropic)	0.355	recovers truth

The truth throughout was 0.35. The through-line: MR removes confounding by construction, but it does not remove pleiotropy. Confounding is the problem it solves; pleiotropy is the problem it introduces, and dealing with pleiotropy is what separates a credible MR analysis from a misleading one.

7.2 A checklist for the next MR paper you read

When you meet an MR result (in a journal, a preprint, or a job talk), these are the questions the preceding chapters equip you to ask. They echo the published guidance for appraising MR studies (Davies, Holmes, and Davey Smith 2018).

Six questions to interrogate any MR estimate

Are the instruments strong? Look for the F-statistic (Chapter 3). Below ~10, weak-instrument bias creeps in, and in the one-sample design it points toward the confounded observational estimate.
Is the exposure association real, and in the right sample? Two-sample MR assumes the exposure and outcome samples don’t overlap. Overlap reintroduces the bias two-sample design was meant to avoid.
How much do the variants disagree? A large Cochran’s $Q$ (Chapter 5) is a symptom of pleiotropy or a mis-specified instrument, and it deserves more than a footnote.
Do the sensitivity analyses agree? IVW, MR-Egger, and the weighted median rest on different assumptions (Chapter 6). Concordance is reassuring; divergence means the headline number is fragile, and the paper should say so.
What does the MR-Egger intercept show, and is it powered? A non-zero intercept flags directional pleiotropy, but a non-significant one is not a clean bill of health (that intercept test is weak).
Is the causal question even coherent? MR estimates a lifelong, average effect of a genetically-proxied exposure. That is not always the same as the effect of an intervention that changes the exposure briefly, in adults, at one dose.

7.3 The limitations worth stating out loud

No method earns trust by hiding its failure modes, and MR has real ones.

Pleiotropy can never be fully excluded. The exclusion restriction is untestable. Sensitivity analyses bound the damage; they do not banish it.
The estimand is peculiar. A variant nudges the exposure by a little, from birth. Extrapolating that to “what happens if a drug halves the exposure in 60-year-olds” is a leap the genetics alone does not license.
Winner’s curse and weak instruments bias effects when the same data both discover and use the variants.
Population structure and assortative mating can quietly break the independence assumption that the whole natural-experiment story depends on.
Canalisation and developmental compensation mean the body may absorb a lifelong genetic nudge in ways an acute intervention would not.

None of this makes MR worthless. It makes MR a tool with a shape: powerful for the questions that fit it, misleading for the ones that don’t. Knowing which is which is the actual expertise, and it is why the honest MR papers spend as much space on assumptions as on estimates.

7.4 Where to go next

The natural extensions build directly on this foundation: multivariable MR (several exposures at once), the mode-based estimator (Hartwig, Davey Smith, and Bowden 2017), MR-PRESSO (formally detecting and removing outlier variants), colocalisation (checking whether the exposure and outcome signals share a causal variant or only happen to be neighbours), and non-linear MR. Each is a variation on the same core move you have now implemented by hand: instrument the exposure with genetics, and reason carefully about what could sit on the path from variant to outcome.

The complete source for everything here is in the repository. Clone it, change TRUE_THETA, add your own broken instruments, and watch the methods respond. Breaking the simulation on purpose is the fastest way to understand what the estimators are really doing.

References

Bowden, Jack, George Davey Smith, and Stephen Burgess. 2015. “Mendelian Randomization with Invalid Instruments: Effect Estimation and Bias Detection Through Egger Regression.” International Journal of Epidemiology 44 (2): 512–25. https://doi.org/10.1093/ije/dyv080.

Bowden, Jack, George Davey Smith, Philip C. Haycock, and Stephen Burgess. 2016. “Consistent Estimation in Mendelian Randomization with Some Invalid Instruments Using a Weighted Median Estimator.” Genetic Epidemiology 40 (4): 304–14. https://doi.org/10.1002/gepi.21965.

Burgess, Stephen, Adam Butterworth, and Simon G. Thompson. 2013. “Mendelian Randomization Analysis with Multiple Genetic Variants Using Summarized Data.” Genetic Epidemiology 37 (7): 658–65. https://doi.org/10.1002/gepi.21758.

Davey Smith, George, and Shah Ebrahim. 2003. “’Mendelian Randomization’: Can Genetic Epidemiology Contribute to Understanding Environmental Determinants of Disease?” International Journal of Epidemiology 32 (1): 1–22. https://doi.org/10.1093/ije/dyg070.

Davies, Neil M., Michael V. Holmes, and George Davey Smith. 2018. “Reading Mendelian Randomisation Studies: A Guide, Glossary, and Checklist for Clinicians.” BMJ 362: k601. https://doi.org/10.1136/bmj.k601.

Hartwig, Fernando Pires, George Davey Smith, and Jack Bowden. 2017. “Robust Inference in Summary Data Mendelian Randomization via the Zero Modal Pleiotropy Assumption.” International Journal of Epidemiology 46 (6): 1985–98. https://doi.org/10.1093/ije/dyx102.

# Takeaways: reading MR critically ```{r} #| label: setup #| include: false source("R/theme_mr.R") source("R/simulate.R") source("R/mr_estimators.R") library(ggplot2) ``` We have built the standard MR toolkit from scratch and watched it recover a causal effect we planted, then defend that effect against pleiotropy. The last and most useful skill is turning the method around: using what you now know to judge an MR analysis you did not run. ## What we established ```{r} #| label: recap #| echo: false sim <- simulate_mr(pleiotropy = "none") dir <- simulate_mr(pleiotropy = "directional") one <- tsls(sim$G, sim$x, sim$y) tsn <- split_two_sample(sim); tsd <- split_two_sample(dir) sx <- \(ts) sapply(seq_len(ncol(ts$sample1$G)), \(j) snp_assoc(ts$sample1$G[,j], ts$sample1$x)["beta"]) sy <- \(ts) sapply(seq_len(ncol(ts$sample2$G)), \(j) snp_assoc(ts$sample2$G[,j], ts$sample2$y)["beta"]) se <- \(ts) sapply(seq_len(ncol(ts$sample2$G)), \(j) snp_assoc(ts$sample2$G[,j], ts$sample2$y)["se"]) ivw_n <- mr_ivw(sx(tsn), sy(tsn), se(tsn)) ivw_d <- mr_ivw(sx(tsd), sy(tsd), se(tsd)) egg_d <- mr_egger(sx(tsd), sy(tsd), se(tsd)) data.frame( Method = c("Naive OLS", "2SLS (one-sample)", "IVW (valid)", "IVW (pleiotropic)", "MR-Egger (pleiotropic)"), Estimate = round(c(naive_ols(sim$x, sim$y)$estimate, one$estimate, ivw_n$estimate, ivw_d$estimate, egg_d$estimate[2]), 3), Verdict = c("biased by confounding", "recovers truth", "recovers truth", "biased by pleiotropy", "recovers truth") ) ``` The truth throughout was `r TRUE_THETA`. The through-line: MR removes *confounding* by construction, but it does not remove *pleiotropy*. Confounding is the problem it solves; pleiotropy is the problem it introduces, and dealing with pleiotropy is what separates a credible MR analysis from a misleading one. ## A checklist for the next MR paper you read When you meet an MR result (in a journal, a preprint, or a job talk), these are the questions the preceding chapters equip you to ask. They echo the published guidance for appraising MR studies [@davies2018reading]. ::: {.callout-tip title="Six questions to interrogate any MR estimate"} 1. **Are the instruments strong?** Look for the F-statistic (Chapter 3). Below ~10, weak-instrument bias creeps in, and in the one-sample design it points toward the confounded observational estimate. 2. **Is the exposure association real, and in the right sample?** Two-sample MR assumes the exposure and outcome samples don't overlap. Overlap reintroduces the bias two-sample design was meant to avoid. 3. **How much do the variants disagree?** A large Cochran's $Q$ (Chapter 5) is a symptom of pleiotropy or a mis-specified instrument, and it deserves more than a footnote. 4. **Do the sensitivity analyses agree?** IVW, MR-Egger, and the weighted median rest on *different* assumptions (Chapter 6). Concordance is reassuring; divergence means the headline number is fragile, and the paper should say so. 5. **What does the MR-Egger intercept show, and is it powered?** A non-zero intercept flags directional pleiotropy, but a non-significant one is not a clean bill of health (that intercept test is weak). 6. **Is the causal question even coherent?** MR estimates a lifelong, average effect of a genetically-proxied exposure. That is not always the same as the effect of an intervention that changes the exposure briefly, in adults, at one dose. ::: ## The limitations worth stating out loud No method earns trust by hiding its failure modes, and MR has real ones. - **Pleiotropy can never be fully excluded.** The exclusion restriction is untestable. Sensitivity analyses bound the damage; they do not banish it. - **The estimand is peculiar.** A variant nudges the exposure by a little, from birth. Extrapolating that to "what happens if a drug halves the exposure in 60-year-olds" is a leap the genetics alone does not license. - **Winner's curse and weak instruments** bias effects when the same data both discover and use the variants. - **Population structure and assortative mating** can quietly break the independence assumption that the whole natural-experiment story depends on. - **Canalisation and developmental compensation** mean the body may absorb a lifelong genetic nudge in ways an acute intervention would not. None of this makes MR worthless. It makes MR a tool with a shape: powerful for the questions that fit it, misleading for the ones that don't. Knowing which is which is the actual expertise, and it is why the honest MR papers spend as much space on assumptions as on estimates. ## Where to go next The natural extensions build directly on this foundation: multivariable MR (several exposures at once), the mode-based estimator [@hartwig2017mode], MR-PRESSO (formally detecting and removing outlier variants), colocalisation (checking whether the exposure and outcome signals share a causal variant or only happen to be neighbours), and non-linear MR. Each is a variation on the same core move you have now implemented by hand: instrument the exposure with genetics, and reason carefully about what could sit on the path from variant to outcome. The complete source for everything here is in the [repository](https://github.com/Lexiyao/mendelian-randomisation-from-scratch). Clone it, change `TRUE_THETA`, add your own broken instruments, and watch the methods respond. Breaking the simulation on purpose is the fastest way to understand what the estimators are really doing. ## References {.unnumbered} ::: {#refs} :::