Figure 2.2 SScP-500 closing data with regression using terms raised to the fifth power.
| (2.1) (2.2) |
Est Close = C0 + (C1 x D)
Est Close=C0+(C1*D)+(C2*D2)+(C3*D3)+(C4*D4)+C5*D5)
Table 2.2 illustrates several interesting features about curve-fitting a data set. First, observe that the value of the constant Co is approximately the same for each equation. This implies that the simplest model, the constant C0, captures a substantial amount of information in the data set.
Then, notice that the absolute value of the constants decreases as the order of the term increases. In other words, in absolute value, Co is greater than C1, which is greater than C2 and on down the line. Therefore, the relative contribution of the higher-order polynomial terms becomes smaller and smaller. However, as you add the higher-order polynomial terms, the line takes on greater curvature and fits the data more closely, as seen in Figures 2.1 and 2.2.
20 Principles of Trading System Design
Table 2.2 Comparison of linear regression coefficients
| C0 | C1 | C2 | C3 | C4 | Cs | |
| Equation2.1 | 560.0865 | 0.537870 | ||||
| Equation2.2 | 570.2379 | -1.94509 | 0.131279 | -0.00154 | -0.00003 | 0.0000006 |
This exercise illustrates many important ideas. First, any model you build for the data should be as simple as possible. In this case, the simple linear regression, with a slope and intercept, captured essentially all the information in the data. Second, adding complexity by adding higher-order terms (read rules) does improve the fit with the data. Thus, we pick up nuances in the data as we build more complex models. The probability that these nuances will repeat exactly is very small. Third, the purpose of our models is to describe how prices have changed over the test period. We used our data to directly calculate the linear regression coefficients. Thus, our model is hostage to the data set. There is no reason why these coefficients should accurately describe any future data. This means that over-fitted trading systems are unlikely to perform as well in the future.
Another example, a variant of the moving-average crossover system, illustrates why it makes sense to limit the number of rules. In the usual case, the dual moving average system has just two rules. For example, for the long entry the 3-day average should cross over the 65-day average and vice versa.
Now, consider a variant that uses more than two averages. For example, buy on the close if both the 3-day and the 4-day moving averages are above the 65-day average. Since there are two "short" averages, this gives us four rules, two each for long and short trades. Using more and more "short" averages rapidly increases the number of rules. For example, if the 3-, 4-, 5-, 6-, and 7-day moving averages should all be above the 65-day average for the long entry, ten rules would apply.
Consider 10 years of Swiss franc continuous contract data, from January 1, 1985, through December 31, 1994, without any initial stop, but allowing $100 for slippage and commissions. The number of rules is varied from 2 to 128 to explore the effects of increasing the number of rules. As the number of rules increases, the number of trades decreases, as shown in Figure 2.3. This illustrates the fact that as you increase the number of rules, you need more data to perform reliable tests.
Rule 2: A Small Number of Rules21