Testing the statistical significance of patterns discovered
It is not clear how to test statistical significance for arbitrary performance metrics. As a proxy, we test the statistical significance of the accuracy of predictions made by component rules.
We use the binomial distribution to compute z-scores under the null hypothesis that a rule predicts randomly on which days its target fund rises.
- Let p be the probability that a fund rises and let n be the number of days for which a rule fires.
- Under the null hypothesis, the expected number of correct predictions is np with standard deviation (np(1-p)1/2
The observed accuracy percentages are: Japan: 57 firings, 63% wins, zscore 2.7 Gold: 187 firings, 63% wins, zscore 4.2 S&P500: 209 firings, 62% wins, zscore 1.8 The second rule has the highest z-score because it targets the gold fund,