Placebo / permutation test

Shuffle the data. Randomize the trade order, scramble the entry signal, then rerun the exact same strategy on the scrambled version. If it makes just as much money on noise as it did on the real series, the original result wasn't a strategy. It was a coincidence wearing a strategy's clothes. That's a permutation test, and it's the placebo control most published backtests skip entirely.

The logic is borrowed straight from clinical trials. A drug has to beat a sugar pill, not just beat nothing. A trading rule has to beat a randomized version of itself, not just beat a buy-and-hold line on a chart. Randomizing the trade order or the bar sequence destroys any real timing relationship between the rule and the market while leaving the underlying return distribution untouched. If performance barely moves, the rule was never reading the market. It was reading the fit.

We run this on everything that survives the earlier filters, as a permanent gate in the pipeline. A meaningful share of strategies that looked genuinely profitable collapse the moment their timing gets shuffled, because the pattern a developer thought they'd found lived inside the parameters, not inside the price action. That's a close cousin of overfitting, where the curve was tuned to history instead of discovering something that generalizes. It also catches cases that plain edge-checking misses: a strategy can clear the bar for no real edge on raw entries and still fail once its exact sequencing gets scrambled.

The test is unglamorous. It doesn't reward a strategy for being clever, only for surviving an attack on its own premise, and it makes a lot of pretty equity curves disappear. That's exactly why it belongs in the pipeline. The more parameters a system carries, and the more variations a developer tried before settling on "the one," the more likely a permutation test is the only thing standing between a real edge and noise dressed up as discipline. That risk compounds with multiple testing: the more shuffled or tweaked configurations you're willing to try, the more one of them will look great by pure chance, and only a placebo control catches it after the fact.

Cheap to run, brutal to fail. If a strategy can't beat a randomized version of itself, it isn't a system. It's a story someone told about a chart, after the chart already happened.