Treatment Effect Estimation Benchmarks
This bundle contains 4 well known and established causal inference benchmark datasets in order to evaluate the performance of causal/treatment effect estimation methods. These datasets are: IHDP, Jobs, Twins and News. All datasets are already publicly available. This bundle merely collects them in a single location for ease of replication.
IHDP is based on Infant Health Development Program (IHDP) clinical trial. Goal: predict the effect of receiving specialised childcare on cognitive test score of the infants. Introduced by .
Jobs combines data from the National Supported Work Program and the Panel Study of Income Dynamics. Goal: predict the effect of job training on employement status. Introduced by .
Twins consists of twin births in the US between 1989-1991. Goal: predict the effect of higher body mass on mortality. This specifically pre-processed data come from .
News is a collection of news articles represented as bags of words. Goal: predict the effect of device type used to read the article on the user experience. Introduced by .
See respective references for more details about the datasets.
 J. L. Hill, ‘Bayesian Nonparametric Modeling for Causal Inference’, Journal of Computational and Graphical Statistics, vol. 20, no. 1, pp. 217–240, Jan. 2011, doi: 10.1198/jcgs.2010.08162.
 F. D. Johansson, U. Shalit, and D. Sontag, ‘Learning representations for counterfactual inference’, in Proceedings of the 33rd International Conference on International Conference on Machine Learning - Volume 48, in ICML’16. New York, NY, USA: JMLR.org, Jun. 2016, pp. 3020–3029.
 C. Louizos, U. Shalit, J. M. Mooij, D. Sontag, R. Zemel, and M. Welling, ‘Causal Effect Inference with Deep Latent-Variable Models’, Advances in Neural Information Processing Systems, vol. 30, 2017, Accessed: May 25, 2021. [Online]. Available: https://proceedings.neurips.cc/paper/2017/hash/94b5bde6de888ddf9cde6748a...
 J. A. Smith and P. E. Todd, ‘Does matching overcome LaLonde’s critique of nonexperimental estimators?’, Journal of Econometrics, vol. 125, no. 1–2, pp. 305–353, 2005.
Please visit the following GitHub repository for further instructions and examples on how to load and use the datasets with Python programming language.
- IHDP dataset IHDP.zip (31.82 MB)
- Jobs dataset JOBS.zip (105.68 kB)
- Twins dataset TWINS.zip (9.54 MB)
- News dataset NEWS.zip (57.18 MB)
- Load IHDP dataset. ihdp.py (7.97 kB)
- Load Jobs dataset. jobs.py (10.40 kB)
- Load Twins dataset. twins.py (6.76 kB)
- Load News dataset. news.py (9.59 kB)
- Evaluation metrics. eval.py (10.11 kB)