We next test the hypothesis that if the optimization period were closer to the actual trading period, the predictions would be more reliable. However, as Tables 3.8 and 3.9 show, there is again no way to predict what the model will do in the succeeding periods. This should be expected because there is no cause-and-effect relationship between our optimized model and market forces. Since we are merely fitting a model to past data, we are not capturing all the fundamental and psychological forces driving the market. Our poor ability to predict the future based only on past price data is not surprising.
Let us carry our argument one step forward. Because we do not capture any cause-and-effect relationships, optimization on one market should have little or no benefit for trading other markets. Indeed, as Table 3.10 shows, optimizing a system on one market (here the deutsche mark) does little to improve performance in other markets.
Table 3.8 Data showing that bringing the optimization period closer to the trading period (11 /88-11 /89) does not predict future performance
Length of Optimized 3 mo. 1990 6mo.1990 9mo.1990 12mo.1990 SMA Profit Profit Profit Profit Profit (Pays) (S)_____($)_____($)_____($)_____($)
20 3,525 -1,625 -1,000 2,650 2,438 25 5,225 -1,900 -2,575 400 -^13 30 4,250 5,338 4,713 7,688 8,475 35 ' 513 5,338 4,713 7,213 8,000 40 63 5,338 4,437 6,213 8,813 45 -2,800 5,338 3,138 4,913 7,638 50 -1,525 5,338 913 2,688 5,413
To Optimize or Not to Optimize?51
Table 3.9 Data showing that relative rankings over recent past (11 /88-11 /89) do not predict future relative ranks
| Length of SMA | Optimized Relative | 3 mo. 1990 Relative | 6 mo. 1990 Relative | 9 mo. 1990 Relative | 12mo. 1990 Relative |
| (Days) | Rank | Rank | Rank | Rank | Rank |
Any optimization exercise has many potential benefits. The first benefit is recognition of the type of market conditions under which the trading system is unprofitable. For any rules that you can construct, you can find market action that produces losses. This happens because the market triggers the signal, and then does just the opposite instead of following through.
The second benefit is verification of the general ideas underlying the model. For example, you can check to see if the model is profitable in trending markets or trendless markets. You have designed the rules to be profitable under certain market assumptions. The optimization exercise allows you to verify if your broad assumptions are correct.
A third benefit is understanding the effect of initial money management stops. You can quantify what level of initial stop allows you to
Table 3.10 Data showing that optimization over one market does not predict performance in other markets
| Deutsche | |||||
| Length of SMA | Mark 11/88-11/89 Profit | Japanese Yen 11/90-7/95 Profit | Cold 11/90-7/95 Profit | Coffee 11/90-7/95 Profit | Heating Oil 11/90-7/95 Profit |
| (Days) | ($) | (S) | (S) | ($) | (S) |
| 3,525 | 8,188 | -16,190 | 30,956 | -26,771 | |
| 5,225 | 7,838 | -15,370 | 29,206 | -21,938 | |
| 4,250 | 8,938 | -13,920 | 40,781 | -21,230 | |
| 7,013 | -10,860 | -5,013 | -18,028 | ||
| 3,963 | -11,400 | -6,343 | -14,316 | ||
| -2,800 | 3,250 | -7,940 | 6,188 | -18,873 | |
| -1,525 | 11,245 | -8,310 | 6,625 | -13,773 |
52 Foundations of System Design
capture the majority of potential profits. For example, if your stop is too wide, your losing trades will be relatively large. On the other hand, if your stop is too close to the starting position, you will be stopped out frequently. Your loss per trade will be small. However, the higher frequency of losing trades means your total drawdown could exceed a larger initial stop.
The biggest benefit of optimization is reinforcing your beliefs about a particular trading system. Ultimately, it is more important for you to implement the trading system exactly as planned. Hence, any testing you do that allows you to understand system performance and become more comfortable with its profit and loss characteristics will help you to execute it with greater confidence in actual trading.
The main point of this section is that you cannot assume your system is going to be as profitable in the future as it has been in the past. This raises the issue of how you control your risks to cope with uncertain future performance. The next section presents risk-control ideas.