Tool · overfitting detector

Is that Sharpe ratio a discovery, or the best draw of a search?

A backtest is the winner of every variant its author tried. Four published tests ask how much of its Sharpe ratio survives that search: the deflated Sharpe ratio, the minimum track record length, the probability of backtest overfitting and the Harvey and Liu haircut. Enter a result, or load the returns of your trials. Everything is computed on this page, and the worked example is the Monitor's own family of strategies.

Runs in your browser. A file you load is read on this device and never sent anywhere.

The selected strategy and its search
Frequency of the returns

The deflated Sharpe turns on this number. Default: the dispersion measured across the Monitor's strategies (worked example below), not the 1/√T null value, which makes the test lenient.

Deflated Sharpe ratioBailey and López de Prado (2014)
 

Probability of real skill as the search widens. The hurdle moves, the Sharpe ratio does not.
Minimum track record lengthBailey and López de Prado (2012)
 to beat zero
 to beat the search's hurdle

Years of track record needed, by Sharpe ratio, for this skewness and kurtosis.
Haircut Sharpe ratioHarvey and Liu (2015)
 

Adjustmentp-valueHaircut SharpeHaircut
The haircut is not a flat 50%: it shrinks as the Sharpe ratio grows (BHY, this search).
Probability of backtest overfittingBailey, Borwein, López de Prado and Zhu (2017)
 

Needs the returns of every trial: load a CSV with several columns, or draw a search on noise.

Worked example

The Monitor's strategies, read as one search

Every strategy the Monitor tests is a trial of one search: what is the best of them worth once the search is counted? Data from the Monitor's export, read by this page.

Account
Currency
Net Sharpe ratios of the family

Loading the Monitor's data.

The family's best, deflated

Overfitting of the selection

Haircut of the bestHarvey and Liu (2015)

Adjustmentp-valueHaircut SharpeHaircut

Methods

What is computed, and from where

Deflated Sharpe ratio

The probabilistic Sharpe ratio against the Sharpe ratio a search of zero-skill trials is expected to find as its best (eq. 1 and 2 of the paper). The Sharpe ratios are per period; the kurtosis is the raw fourth moment (3 for a normal law). is the variance of the trials' Sharpe ratios: the test turns on it. Its replacement by the variance of one Sharpe estimate under the null, , sets the hurdle far too low on long samples and is not offered here.

Correlated trials count for fewer independent ones: (appendix A.3, eq. 9), with the average pairwise correlation (eq. 8).

Minimum track record length

The number of observations after which the observed Sharpe ratio exceeds a threshold at the chosen confidence, given the skewness and kurtosis. Against zero it ignores the search; against the deflated hurdle it does not. Both are shown.

Haircut Sharpe ratio

The single-test p-value of (monthly units, a Student t with degrees of freedom) is adjusted for tests by Bonferroni, Holm and Benjamini, Hochberg and Yekutieli, then turned back into a Sharpe ratio. Holm and BHY need the p-values of the other tests: as in Harvey and Liu's program, they are simulated 5,000 times from the model of Harvey, Liu and Zhu (2016), whose parameters are interpolated at the average correlation, and the median is kept. The autocorrelation correction is Lo's (2002). Their program counts a daily year as 360 days; this page uses 252.

Probability of backtest overfitting

Combinatorially symmetric cross-validation: the matrix of returns is cut into blocks; for each of the ways of choosing half of them as the training set, the trial with the best training Sharpe is ranked on the other half. The PBO is the share of splits where that winner lands below the median. At that is 12,870 splits (the paper prints 12,780). Computed in a Web Worker.

Checked against Python

The page's code (stats.js) is tested against the Python of our research library on fixed fixtures (the strict deflated Sharpe and the CSCV reference implementation), to within 1e-12, and against every worked example printed in the four papers.

References

  1. Bailey, D. H. and López de Prado, M. (2014). The Deflated Sharpe Ratio: Correcting for Selection Bias, Backtest Overfitting, and Non-Normality. Journal of Portfolio Management 40(5), 94–107. SSRN 2460551
  2. Bailey, D. H., Borwein, J. M., López de Prado, M. and Zhu, Q. J. (2017). The Probability of Backtest Overfitting. Journal of Computational Finance 20(4), 39–69. SSRN 2326253
  3. Harvey, C. R. and Liu, Y. (2015). Backtesting. Journal of Portfolio Management 42(1), 13–28. SSRN 2345489
  4. Bailey, D. H. and López de Prado, M. (2012). The Sharpe Ratio Efficient Frontier. Journal of Risk 15(2), 3–44. SSRN 1821643
  5. Harvey, C. R., Liu, Y. and Zhu, H. (2016). … and the Cross-Section of Expected Returns. Review of Financial Studies 29(1), 5–68.
  6. Lo, A. W. (2002). The Statistics of Sharpe Ratios. Financial Analysts Journal 58(4), 36–52.

Research output, not investment advice.