The optimizer went through hundreds of combinations, the best one showed 300% in history — and in real trading, the account went into negative territory. Analyzing the re-optimization: how the strategy learns noise instead of patterns, by what signs it is visible (the result is at a peak instead of a plateau, few transactions, an ideal curve) and how to check honestly — out-of-sample test, walk-forward, resistance to parameters.
A familiar scenario: the optimizer went through several hundred combinations of parameters, and the best one showed 300% in history. The strategy goes into real trading, and after a couple of weeks, the account is in the red. The test did not "break" and the market did not "go crazy". Most likely, this is a re-optimization.
What is overoptimization?
The history of the market consists of patterns and noise. When we go through many parameters in the same section of history, sooner or later we find a combination that fits perfectly not on a pattern, but on noise — on the random coincidences of this particular piece of data.
Such a strategy does not "know" anything about the market. She has memorized the past, and she crumbles on any new data. The more parameters and the more runs of the optimizer, the higher the chance of getting exactly this result.
Signs that the strategy has been adjusted
- The result lives on a peak, not on a plateau. The best set of parameters gives 300%, and the neighboring one (slightly shifted value) is already -20%. The real pattern is resistant to small changes in parameters; noise is not.
- Few deals. On 30-50 trades, statistics mean almost nothing: a few lucky accidents make any curve beautiful.
- All the profitability is on a couple of trades. You take away one or two of the best ones, and nothing remains of the result.
- The curve is too perfect. Steady growth with almost no drawdowns in a volatile market is a reason not to rejoice, but to be wary.
- There is no answer "why it works". If a strategy does not have a clear reason to earn money (the behavior of participants, the structure of the market, the costs of others), most likely, it does not exist.
How to check the strategy honestly
Out-of-sample test. Basic hygiene: The story is divided into two parts. On the first one, the strategy is adjusted, on the second one it is only checked, without the right to twist anything. The result of the second part is an honest assessment.
Walk-forward. A more rigorous version: the window "configured in this section — checked in the next one" slides through the entire history. The strategy repeatedly encounters data that it "hasn't seen." If the result varies from window to window, this is a sign of a pattern, not a fit.
Resistance to parameters. Look not only at the best set, but at the surroundings: if small parameter shifts do not ruin the result, you have found a plateau, not a random peak.
Fair simulation conditions. Fees, slippage, fundings, and liquidity. Without them, even the right methodology checks the strategy in a fairy—tale world - and it has to be traded in the present.
Where is it already embedded?
We built XTester around exactly this problem: the tester should help us catch the fit, not help us ignore it. The separation of data for out—of-sample verification, honest cost accounting in simulation, and metrics that show the stability of the result are part of the engine, not an add-on.
Early access is open: xtester.pw .
Frequent questions
How much history do I need for the test?
There is no single figure — the number of transactions and the variety of market conditions are more important.: The strategy must go through both trend, sideways, and high volatility. A hundred or two trades in different market modes says more than a thousand candlesticks in one quiet year.
Which is better: a simple split of the story or a walk-forward?
Separation is a necessary minimum, and walk-forward is a more rigorous test of stability over time. If the strategy claims to be real money, the walk-forward is worth the time.
The strategy failed the out-of-sample test. Should I fix it or throw it away?
Be careful: if you "fix" by tweaking the parameters until it passes, this is the same re—optimization, only manually. It's worth returning to the hypothesis: why the strategy should work at all. If there is no answer, there is no strategy.
