If you have a computer, you can easily set up a search to find the "optimum" values for a system over historical data. The results can be truly astonishing. Imagine your profits if you could only have known ahead of time what the most profitable parameter combination was going to be. Therein lies the rub. The unfortunate fact is that parameters that work best on past data rarely provide similar performance in the future.
The term "optimization" is used rather loosely here to include all the activities affecting selection of parameter values in a trading system. We have already seen the difficulties of curve-fitting a model. You can also consider lower levels of optimization, in which you test variables over a broad range of values and markets, and try to select the one you like "best." But the real issue is not whether a particular set is the best. It is whether you believe sufficiently in the system to trade it without deviations. The primary benefit of optimization may be that you improve your comfort level with a particular system.
The problem with system optimization is that past price patterns do not repeat exactly in the future. The same is true of intermarket relationships. Although broad relationships follow from historical data, there can be differences in the time-lags between events and the relative magnitudes of the effects.
You must also resolve other conflicts. For example, you must choose the period you will use to optimize your trading system values. As you will quickly discover, the values you choose depend on the length of the test period. You must also determine how often you will reop-timize your system in the future. You must then prescribe the time for which the optimized values are valid.
For example, you may decide to use 3 years of data to optimize the values and recalculate them after 3 months. Thus, one solution may be to reoptimize after 3 months on the latest 3 years of data available. This is equivalent to retraining your favorite neural net. If you do reoptimize, you must determine how to treat trades that may be open from the previous period or values of the trading system.
To Optimize or Not to Optimize?49
You must also decide if you want to use the same values of your system parameters on all markets. If not, you will have to optimize the system on each market separately. In that case, you must keep up a program of reoptimization and recalibration for each of your systems over every market that you trade. Is all this effort worth the trouble? The results of deterministic testing do not support any attempts at finding the "best" or optimized variables.
Consider the following test using actual deutsche mark futures contracts. The rollover dates are the twenty-first day of the month before expiration. For simplicity, we will trade just one contract, allowing $100 for slippage and commissions, with a $1,500 initial money management stop. We will use a variation of the moving average crossover system, trading not the crossover, but a 5-day breakout in prices after the crossover. Thus, if the shorter moving average was above the longer moving average, then a 5-day breakout above the highs would trigger a long entry. Also included is a simple exit condition, ending the trade on the close of the twentieth day in the trade. One attractive feature of this arbitrary system is that the lengths of the short and long moving average can be optimized.
The calculations are simplified by fixing the length of the short average to a 3-day simple moving average of the close. The length of the longer simple moving average varies from 20 to 50 days, with an increment of 5 days. The test period was from November 14, 1983, through November 21, 1989. The performance of the various models was observed 3, 6, 9, and 12 months into the future. As Tables 3.6 and 3.7 show, there is no predicting how the model will do over a future period. The relative rankings change from period to period without any pattern or consistency.
Table 3.6 Data showing that past performance does not predict future performance
Length of Optimized 3 mo. 1990 6mo.1990 9 mo. 1990 12 mo. 1990 SMA Profit Profit Profit Profit Profit (Days) ($)______(S)______($)______($)______(S)