Why Back Testing is Better than Forward Testing

[ForexArticles]

Firstly back-testing should be done as accurately as possible. This means that all the best practices of back-testing should be implemented. Some examples are:

Including reasonable spreads
Avoiding look forward bias
Including any possible forms of commissions if applicable
Using historical data which is as accurate and complete as possible
Ensuring trade logic tested is reasonable and practicable in actual trading
Reducing slippage (although this is normally not a problem with Forex due to its immense liquidity)
With accurate back-testing, characteristics of the trading strategy can be observed and analysed reliably. What this means is the risk and returns profile of the strategy is statistically reliable, and any performance is deemed to be attributed to the strategy and not due to pure chance.

Forward testing is not the most efficient manner to determine risk and returns characteristics of a strategy. A forward test is one price series in time. Backtesting is also one price series in time. There is no significant difference – let’s say you forward test for 3 months to make conclusions, won’t it become a 3 month back test at the point of conclusion?

In addition, backtesting can be done over multiple random price series for a test of robustness. This might be done viaMonte Carlosimulation or Boot strapping.

Next, a test of 5 years to cover a full market cycle is required. This cannot be achieved by forward testing. If you are running a strategy on forward testing and it generates 10 trades over a period of 2 months, that’s hardly statistically significant to make a reliable conclusion about the risk and returns characteristics of the strategy. These 10 trades could be due to pure chance. Many more trades are required to be statistically significant and this might be years of forward testing. Back testing is much more efficient.

Having said that, the purpose of forward testing is to test for operational issues such as whether your IT infrastructure is sufficient and stable for live trading, whether the broker is able to fill your order properly, and whether a forex trading software has any bugs etc. You will also get a feel of the robot’s behaviour and also gain confidence that the robot operates as it should.

Back Testing Tip 1: Time Frame
Choice of a suitable time frame must be aligned to the type of strategy to be traded.

Using a shorter time frame (e.g. 5min, 15min) introduces more noise into the price series making it difficult to find the trend. It is more suitable for scalping, short term arbitrage, mean reversion type of strategies in general. Shorter time frame responds quickly to price changes, results in shorter holding periods, and likely more trades are generated. Statistically the more trades generated, the closer actual mean returns match “population mean returns” or “theoretical mean returns”.

In simpler coin toss analogy, even when toss of the coin is statistically 50-50, there can be 10 heads in a row for 10 tosses of the coin. However when we toss 1,000 times, the proportions of heads should lean towards 50%. This same concept applies to trading. We need more observations of trades to determine the actual risk and returns of the system. Shorter time frames are more sensitive to execution quality where slippage matters more.

Using a longer time frame (e.g. weekly) puts more emphasis on the trend and less emphasis on other components such as mean reversion. Longer time frames also react slower to price changes. Longer time frame is not as sensitive to execution quality and slippage rarely matters.

Back Testing Tip 2: Testing
Good trading strategies must be based on sound logic or economics. This should be the base case for trade method ideas generation. Backtesting is then the only method to validate trade ideas generated. Backtesting is often misunderstood to be a tool to “discover” trading strategies by scanning a massive combination of techniques or indicators. The base case for backtesting is actually to validate ideas to seek empirical evidences about a hypothesis (in this case to determine if a strategy is statistically deemed to be profitable beyond mere chance).

That said, data mining techniques to discover strategies are available. However, we need to be aware to discount data mined strategies’ results to account for a positive bias in data mined results. Backtesting is also to test how robust a strategy is. The strategy should work over different situations such as different price regimes, in a fairly similar or predictable fashion. Backtesting also gives us an indication of expectations, both in terms of risks and returns.

Back Testing Tip 3: Performance Monitoring
After a system is trading live, it is entering a new phase of life. You should at regular intervals monitor trading results and compare with expectations (both risk and returns) of your backtest, to determine if the system is working properly.

However you should definitely not micro manage your system. A good rule of thumb is to monitor the system on a monthly basis and not on a trade by trade basis! So long as the expectations are reasonably similar between actual trades and backtested performance, the system should be performing normally. It is very important to recognise that statistically, trades would be subjected to strings of wins or losses. Do not be overly alarmed.

One simple method to monitor is to check the actual drawdown of your live system and compare it to your maximum drawdown of the backtest. The actual current drawdown exceeding the maximum drawdown tested is a red flag that the system is not performing as well and market conditions may have changed.

At this point it is important to take note that backtesting should be done over a full market cycle (for example 5 years). It is pretty meaningless to backtest for 2 months and expect the live trades to follow the expectations (risk and returns) of the 2 month backtest.

[BubbleGum]

One very important reason that backtesting will never give you an accurate picture is that it leaves out market conditions. It's very easy to look at the past and see all the nice signals that would be profitable.