Publications

---

Critical Values Robust to P-hacking (with Pascal Michaillat)

(previously titled "Incentive-Compatible Critical Values")

Despite its prevalence, p-hacking is not taken into account in classical hypothesis testing theory. To address this problem, we build a model of p-hacking and use it to construct critical values such that, if these values are used to determine significance, and if scientists adjust their p-hacking behavior to these new significance standards, then significant results occur with the desired frequency. Because such robust critical values allow for p-hacking, they are larger than classical critical values. For instance, in the model calibrated to the social and medical sciences, the robust critical value for any test statistic is the classical critical value for the same test statistic with one fifth of the significance level.

Inference on Winners (with Isaiah Andrews and Toru Kitagawa), revise and resubmit at Quarterly Journal of Economics

R Code

2019 Version (referenced in "Inference After Estimation of Breaks")

Many empirical questions concern target parameters selected through optimization.  For example, researchers may be interested in the effectiveness of the best policy found in a randomized trial, or the best-performing investment strategy based on historical data.  Such settings give rise to a winner's curse, where conventional estimates are biased and conventional confidence intervals are unreliable.  This paper develops optimal confidence intervals and median-unbiased estimators that are valid conditional on the target selected and so overcome this winner's curse. If one requires validity only on average over targets that might have been selected, we develop hybrid procedures that combine conditional and projection confidence intervals to offer further performance gains relative to existing alternatives.