How to read a trading probability

If you read one page on this site, read this one. Everything else — day types, volume events, options positioning — is a number attached to a claim, and this is how you check whether the number deserves your attention.

1. Compared to what? The base rate

A base rate is how often something happens with no prediction at all. It is the score you get for knowing nothing, and it is the only meaningful yardstick for a prediction.

Suppose sessions come in five flavours and the most common one shows up 37% of the time. A "model" that ignores the chart and always answers with that one class is right 37% of the time. So when someone tells you a day-type model is 66% accurate, the interesting number is not 66. It is 66 against 37 — roughly 1.8 times the score for knowing nothing.

Change the base rate and the same accuracy tells a completely different story:

This is the single most common way trading statistics mislead, and it is almost never malicious. People quote the accuracy because it is the number their tool printed. The base rate takes extra work to compute, so it goes missing.

Rule: if a figure arrives without its base rate, treat it as unlabelled until you find one. Ask what a coin, a constant guess, or "yesterday's answer" would have scored on the same test.

2. Calibration: does 70% mean 70%?

Accuracy asks how often the top answer was right. Calibration asks something different and more useful: do the numbers mean what they say?

Here is how you test it, and you can do this yourself with a spreadsheet. Collect every occasion the model said "70%". Ignore everything else. Now count how often the thing actually happened in that pile. If it is around 70%, the model is calibrated at that level. Repeat for the 30% bucket, the 50% bucket, the 90% bucket. Plot the promised probability against the observed frequency; a calibrated model sits on the diagonal.

Calibration and accuracy are independent properties, which surprises people:

• A weather forecaster who says "40% chance of rain" every single day, in a place where it rains 40% of days, is perfectly calibrated and completely useless. Zero discrimination — the forecast never varies.
• A model that is right 80% of the time but announces 99% confidence every time is accurate and badly calibrated. Size a position off that 99% and the arithmetic of your risk is wrong even though the calls are good.

You want both. Calibration is what makes a probability usable as an input to anything — a weighting, a position size, a decision about whether to bother — instead of just a ranking.

3. Top-1 versus top-2 coverage

With five possible day types, forcing an answer down to one label throws away most of what the model knows. A distribution like this is common:

Reported as a label, that becomes "trend-up day" — which sounds confident and is 34% likely. Reported honestly it is: a mild upward lean, a live reversal scenario, and near-certainty that the session is not a downtrend. Those two readings lead to different behaviour.

Top-2 coverage is the two highest probabilities added together — how often the truth falls inside the two scenarios you named. It is usually the more decision-relevant number, because planning for two possibilities is a thing a human can actually do.

All four figures are measured on held-out years — calendar years excluded entirely from training, so the model had never seen a single bar of them. Notice the honest shape of the result: the early read is much weaker than the full-day read, because at 90 minutes there genuinely is less to go on. A tool that reports the same confidence at 9:45 as at 15:45 is not measuring anything.

4. Why 52–56% is what real edge looks like

Beginners routinely walk away from a 54% number because a course promised them 90%. That reaction is backwards, and understanding why is most of what separates people who last from people who do not.

Liquid markets are competitive. Thousands of well-capitalised participants are looking at the same data with better tooling than you have. Anything that reliably wins 80% of the time attracts capital until the price moves and it stops winning 80% of the time. What survives is small — and it is small because it survived.

Look at what happens when the big numbers get measured:

Measured against 3,946 S&P 500 E-mini trading days, 2010–2026. The last row is the proportion failing within 5, 10 and 20 bars respectively.

The pattern is consistent and it is worth internalising: the directions are right, the numbers are inflated by roughly 7 to 10 percentage points. The tradition is not lying — these are real phenomena, observed by people who spent decades in front of charts. The numbers are just remembered rather than counted. A useful correction when you hear a classroom probability: believe the direction, discount the number by about ten percent.

And note the "80% rule" row specifically. Under no reasonable definition does breakout failure reach 80%; the ceiling is about 70%, and only if you wait 20 bars. That claim has been repeated for years without anyone publishing the count.

5. Sample size, in plain words

A win rate is an estimate, and estimates wobble. How much they wobble depends almost entirely on how many observations you have.

Flip a fair coin 20 times. Getting 13 heads — a "65% win rate" — happens often enough to be unremarkable. Flip it 20,000 times and 65% never happens. The number did not change; the sample did.

Two standard errors, the usual rough band. This is why a 2-point edge is only publishable on tens of thousands of samples — and why a backtest with 30 trades in it proves nothing whatever its win rate.

Whenever someone shows you a win rate, the very next question is n. If they do not have it, or will not say, you have learned everything you need to know.

6. p-values, in plain words

A p-value answers one narrow question: if there were genuinely no effect at all, how often would pure chance produce a result at least this striking?

• p = 0.05 — about one time in twenty.
• p = 0.002 — about two times in a thousand.
• p = 2e-6 — about two times in a million.

Three things a p-value is not:

• Not the probability the finding is true. It is computed by assuming the opposite — that there is no effect — and asking how surprising the data would be under that assumption.
• Not a measure of size. A 0.3-point edge on a million samples can hit p = 1e-20 and still be too small to survive transaction costs. Effect size and statistical significance are separate questions and you need both.
• Not trustworthy on its own, because of the next section.

The multiple-comparisons trap

Test twenty unrelated ideas at the usual 0.05 threshold and, on average, one comes back "significant" purely by luck. Test two hundred and you get ten. This is the mechanism behind an enormous share of published trading research, and it requires no dishonesty at all — just enthusiasm and a fast computer.

There are three defences, and honest research uses all three:

• Pre-registration. Write down the exact test, the exact threshold and the exact success criterion before running it. This is what makes the difference between a test and a search. If the hypothesis was written after seeing the result, it is not evidence — it is a description.
• Correction for how many things you tried. If you run 12 tests, divide your threshold by 12. Quant Data's cross-system tests used exactly this, giving a threshold of 0.0042 instead of 0.05.
• Held-out data. Evaluate only on years the model never saw. A model scored on data it was fitted to is grading its own homework, and it will always pass.

Quant Data adds one more: a result must hold with the same sign in both halves of the sample period. An edge that only exists in 2010–2017 and vanishes afterwards is a historical fact, not a property of markets.

7. Published failures are the strongest evidence you will get

Here is the uncomfortable asymmetry: anyone can show you their best result. Selecting the winner out of a hundred attempts costs nothing and looks identical to genuine skill from the outside. You cannot tell them apart from the winning number alone.

What you can check is whether the losses are on the table. A research page that only ever shows wins is a selection. So here are Quant Data's, in public:

Read those three cards as the credential. They are the reason to take the 56.5% seriously. A shop that publishes "our flagship signal does not work on Bitcoin, and one of our whole research directions came back at coin-flip" is a shop that counted, and reported the count.

AUC (area under the ROC curve) is a standard way to score a ranking: 0.5 is random, 1.0 is perfect ordering. Anything below about 0.55 on noisy financial data is usually not worth building on.

8. A checklist you can actually use

Next time you meet a trading statistic — here or anywhere — run it through this:

1. Compared to what? What is the base rate? What would a constant guess have scored?
2. How many? What is n? Under a few hundred, the number is decorative.
3. Measured on what? The same data it was built from, or data held out entirely?
4. Written down first? Was the hypothesis pre-registered, or discovered by searching?
5. How many other things were tried? And was the threshold adjusted for them?
6. Where are the failures? If none are published, assume they exist.
7. Does the size matter? Significant and useful are different words.
8. Does it still hold after costs? A 0.25-point edge against a 0.3-point spread is a losing strategy with good statistics.

Next

Educational analytics. Nothing on this page is investment advice or a recommendation to buy or sell anything.