Overfitting / curve-fitting

Tune enough parameters and any strategy will fit the history perfectly, including the random noise that won't repeat. That's curve-fitting: a system optimized to a past that's already gone, dressed up as a discovery. It looks immaculate in-sample and falls apart the moment it meets data it never saw.

The mechanism is simple math working against you. Give an optimizer enough free parameters and it will always find some combination that explains the noise in your one sample, because noise has nothing distinguishing it from a real pattern except luck. Add a stop-loss tweak here, a lookback window there, a filter tuned to catch three good trades from a single year, and the equity curve straightens right out. It hasn't found how markets behave. It has found how that one slice of history happened to move, once, and won't move again.

The tell is fragility. Nudge one setting — the moving average from 20 to 22, the stop from 2% to 2.2% — and the beautiful curve craters. A strategy with a real, structural edge tolerates small changes because the edge doesn't live in any single number. A curve-fit strategy is memorizing, not learning, and a memorized answer is useless against a question it hasn't seen before.

We guard against it by testing out-of-sample: years and instruments the optimizer never touched, held back specifically so the system can't cheat its way to a good score, which is the same logic behind walk-forward testing. It also means a strategy that was run through hundreds of parameter combinations before we saw it has to clear a higher bar than one that wasn't, because the more configurations get tried, the more likely one fits the noise purely by chance — the multiple testing problem sitting inside every optimizer run.

Pure overfit turns out to be a smaller reject bucket than most people expect. Most curve-fit systems never get charged with that specific crime, because they trip an earlier check first. Run the same rules against a placebo test, scrambled or shuffled data, and the "edge" often survives just as well as it did on the real thing — proof there was never a signal to overfit, just a well-fitted mirage sitting on top of nothing.

A strategy with fourteen tuned inputs hasn't found an edge. It's remembering one unrepeatable path through the past, in detail, and the past does not repeat on schedule.