← All strategiesHow we test →

Why do most backtests lie?

And what it takes to find a trading strategy that actually has an edge.

The core problem

When you run a TradingView backtest, it answers the question: did this rule profit on the data it was designed for? That's the wrong question. What you want to know is: does it have a real edge on data it hasn't seen yet?

Those two questions have very different answers — and the gap between them is where almost all apparent strategy edges disappear.

Common reasons backtests fail in live trading

1. In-sample overfitting ("curve-fitting")

If you test enough variations of a strategy on historical data, some will profit — by chance. The more parameters you search, the more likely you are to find a configuration that happens to match past noise. The fix: hold out the most recent data and never touch it during development. The strategy's OOS performance is the only score that counts.

2. Look-ahead bias

A signal that uses today's close to generate today's entry fills at today's close — but you couldn't have known today's close at the time of entry. This is subtler than it sounds; it shows up in indicator calculations, rebalancing rules, and stop-loss placement.

3. Unrealistic transaction costs

TradingView's default commission settings are often too low, and bid-ask spread is frequently ignored entirely. A strategy that trades 200 times a year with a 0.05% spread on each leg has 20% annual drag before alpha. We apply realistic costs and then run a 3× stress test to check robustness.

4. Parameter fragility

A strategy that only works at exactly one parameter setting — say, an EMA crossover at 9/21 but not at 8/20 or 10/22 — is almost certainly overfit. A real edge should be stable across small parameter perturbations. We test ±20% shifts across all parameters.

5. Single-asset data snooping

A mean-reversion rule that works perfectly on EEM might be exploiting a specific regime in emerging market equities, not a general edge. We test each strategy on at least 6 other assets and timeframes. Broad results are more credible than narrow ones.

Why a high Sharpe ratio isn't enough

A Sharpe of 0.7 from 18 months of daily returns has a 95% confidence interval of roughly [0.2, 1.2]. That's not a tight enough range to trade on.

The Probabilistic Sharpe Ratio (PSR) converts a sample Sharpe into a probability: what's the chance the true underlying Sharpe is positive? It accounts for sample size, skewness, and kurtosis. We require PSR > 0.6 — meaning more than 60% confidence the edge is real, not just noise. We also run a moving-block bootstrap (1,000 resamples, p < 0.10) to catch strategies that aligned with a single lucky stretch.

Together these checks eliminate the majority of strategies that would pass a naive Sharpe screen.

What "DEPLOYABLE" actually means

In our catalog, DEPLOYABLE means a strategy passed all five checks simultaneously: positive OOS Sharpe, cost-resistant, PSR ≥ 0.6, bootstrap p < 0.10, and stable under parameter perturbation. It doesn't mean "guaranteed to profit." Markets change; strategies can degrade. DEPLOYABLE means the evidence for an edge is strong enough to trade at small size — with ongoing live monitoring.

Out of 10 common TradingView strategies we tested, 4 passed. Most failed on costs or bootstrap significance.

See the results

We've published full per-strategy reports — equity curves, all the stats, and live forward-tracking on the older ones — for the 10 most popular strategies.

▶ See all 10 verdict reports

Want your strategy tested?

Submit any strategy — a name, a plain-English description, or a TradingView Pine Script — and we'll run it through the same pipeline and publish the verdict.

▶ Request a free analysis

Not investment advice. Full methodology → · All strategies →