The test summary leaves out some important information, highlighted below. You may wish to examine these factors in greater detail.
One simple ratio is the recovery factor (RF). RF is absolute value of the ratio of net profit to maximum intraday drawdown, and it measures how far you recovered from the depth of the drawdown. In Table 3.13 the recovery factor is approximately equal to 5.6 (155,675/27,881). This number should be greater than 2, and the higher the better. It tells whether the potential benefits over the long haul are worth the aggravations caused by the drawdowns.
Another useful value is the adjusted gross profit, in which the largest winning trade is deducted from the gross profit. To penalize the system, do not correspondingly deduct the largest losing trade. The rationale here is not to expect to get the periods with large profits, but that a period of losses comparable to the worst losses in the test period is likely. The profit factor is then recalculated to see if it is still greater than 1. For the data in Table 3.13, the adjusted gross profit is $155,675 - 40,769 = $114,906. The adjusted profit factor is then 114,906/111,244 = 1.03. This is a sharp reduction from the reported profit factor of 2.40. Thus, a more realistic assessment of this system is that it will produce a small net profit over time.
The summary also does not give a histogram of your trades. You may wish to export your data to a spreadsheet to look for the maximum favorable excursion and maximum adverse excursion. These quantities will be explained in chapter 4 with the 65sma-3cc system.
The summary does not give you any feel for the variation in test results. It does not give a standard deviation of trade profits and losses for all trades. The variability is another important item you should calculate, using a spreadsheet if needed. The variation tells you what you can expect for volatility of returns.
You cannot get an idea of how a typical trade evolves in time from the test summary. For example, it does not tell you the average profit or maximum profit or loss on a day-in-trade basis. It does not show what happened on day 1 in the trade, or day 10 in the trade. A typical trade template, by Chande and Kroll, as discussed in The New Technical Trader
A Reality Check71
(see bibliography), would help you understand the time-price evolution of a typical trade.
In addition, the test summary does not give a realistic impact of slippage. The software provides fills in a manner that may not be representative of fills in the real world. It is safer to assume that you will experience greater slippage than the model. In some instances, the software will give you a fill that you could not have obtained in practice. If this happened to be a big winner, you may overestimate trade profitability. Hence, you are better off using the average trade numbers to assess system performance, since they have averaged out the effects over many trades.
The performance summary also does not give any idea of how many successive .v-month periods would have been profitable. For example, it is useful to know how many successive 6-month periods have been profitable over a 5-year period. You could use any time interval you like. This breakdown tells you how quickly you can expect to get out of drawdowns, and is a vital piece of information for your mental approach to trading the system.
The most important factor to recognize is that the test summary does not tell you how the system will perform in the future. Your test results are hostage to your data. You should look below the surface of the results to get a better understanding of your system tests.
Ideally, you should examine the results on a trade-by-trade basis on the charts to understand how your system rules worked. This will reinforce your trading beliefs, and give you a good feel for when the system does or does not work. A study of unprofitable trades often reveals flaws in your logic. Convince yourself that you want to follow this system because its rules make money under market conditions that are likely to repeat in the future. A trade-by-trade review may also strengthen your ability to use discretion in trade entries or exits.