A familiar scenario: the optimizer runs through several hundred parameter combinations, the best showing 300% historically. The strategy goes into live trading — and after a couple of weeks the account is in the red. The test did not "break" and the market did not "go crazy." Most likely, this is overfitting.

What is overfitting

The market history consists of patterns and noise. When we try many parameters on the same historical segment, sooner or later we find a combination that fits not a pattern, but the noise — random coincidences specific to that data segment.

Such a strategy "knows" nothing about the market. It has memorized the past — and falls apart on any new data. The more parameters and optimizer runs, the higher the chance of getting exactly such a result.

Signs that a strategy is overfitted

  • The result lives at a peak, not on a plateau. The best parameter set gives 300%, while a neighboring (slightly shifted value) already shows −20%. A real pattern is stable to small parameter changes; noise is not.
  • Few trades. With 30–50 trades, statistics mean almost nothing: several lucky coincidences make any equity curve look good.
  • All profit comes from a couple of trades. Remove one or two best trades — and nothing remains of the result.
  • Too perfect a curve. Smooth growth almost without drawdowns in a volatile market is a reason not to rejoice, but to be cautious.
  • No answer to "why does it work." If a strategy has no clear reason to make money (participant behavior, market structure, competitors' costs), most likely it doesn't.

How to test a strategy honestly

Out-of-sample test. Basic hygiene: history is divided into two parts. The strategy is tuned on the first part, then only tested on the second, without any tweaking. The result on the second part is the honest evaluation.

Walk-forward. A stricter variant: the window "tune on this segment — test on the next" slides through the whole history. The strategy repeatedly encounters data it "hasn't seen." If the result holds from window to window — that's a sign of a pattern, not overfitting.

Parameter stability. Look not only at the best set, but around it: if small parameter shifts do not ruin the result — you found a plateau, not a random peak.

Honest simulation conditions. Commissions, slippage, funding fees, liquidity. Without them, even the correct methodology tests the strategy in a fairy-tale world — but it has to trade in the real one.

Where this is already built-in

We designed XTester around this very problem: the tester should help catch overfitting, not help miss it. Data separation for out-of-sample testing, honest accounting of costs in simulation, metrics that show result stability — these are part of the engine, not an add-on.

Early access is open: xtester.pw.

Frequently asked questions

How much history is needed for the test?
There is no single number — the number of trades and diversity of market conditions matter more: the strategy should pass through a trend, a flat, and high volatility. A hundred or two trades in different market regimes say more than thousands of candles from one calm year.

Which is better: simple history split or walk-forward?
Splitting is the necessary minimum, walk-forward is a stricter check of stability over time. If the strategy is aiming for real money, walk-forward is worth the time spent.

The strategy failed the out-of-sample test. Fix it or discard it?
Be careful: if "fixing" means tweaking parameters until it passes — that is also overfitting, but manually. It's better to return to the hypothesis: why should the strategy work at all. No answer — no strategy.