foxalgo.io

We stopped touching the model in 2017. Then we watched what happened.

13,278 market-sessions later, across 103 months since 2018 — fit on data through 2017, scored on every session since. Two checks, not one number to trust blindly. First: how reliable has the model been on years of out-of-sample data it never saw during fitting. Second: a live, append-only record of this site's own daily predictions, scored after the fact.

Measured odds. No trade calls. what this means →

When we say X%, how often does it happen?

Every grid probability is bucketed by what we predicted (10-point-percentage bins) and compared against the realized rate on out-of-sample data — 2018–2026, data the fitting process never touched (fitting period: 2010–2017). If the model is honest, the realized bar should sit right on top of the predicted bar in every bucket.

FOXALGO / SCORECARD INSTRUMENT 01
n = 164,920,140MAE 0.84 pp

CALIBRATION / PREDICTED VS REALIZED

00252550507575100100perfect calibrationpredicted %realized %
View as table
bucketpredictedrealizederrorn
0-10%3.4%3.5%+0.0 pp56,042,860
10-20%15.1%15.0%-0.1 pp20,652,607
20-30%24.8%24.8%+0.0 pp16,450,897
30-40%34.3%34.3%-0.0 pp11,760,851
40-50%45.2%45.1%-0.1 pp12,768,405
50-60%53.8%53.7%-0.1 pp7,300,159
60-70%65.9%65.8%-0.1 pp9,908,539
70-80%74.0%73.6%-0.3 pp12,967,003
80-90%86.1%86.2%+0.1 pp16,535,975
90-100%90.3%89.9%-0.4 pp532,844
FOXALGO / SCORECARD INSTRUMENT 02
n = 776,797,176MAE 0.49 pp

CALIBRATION / PREDICTED VS REALIZED

00252550507575100100perfect calibrationpredicted %realized %
View as table
bucketpredictedrealizederrorn
0-10%2.4%2.4%+0.0 pp410,478,455
10-20%14.7%15.2%+0.4 pp85,032,162
20-30%24.5%25.3%+0.8 pp46,212,284
30-40%35.0%35.6%+0.7 pp48,007,249
40-50%45.2%46.3%+1.1 pp27,125,949
50-60%55.3%56.2%+0.9 pp38,443,925
60-70%64.0%64.9%+0.9 pp42,457,281
70-80%75.8%76.4%+0.6 pp36,150,469
80-90%83.5%83.9%+0.4 pp42,889,402
FOXALGO / SCORECARD INSTRUMENT 03
n = 30,337,932MAE 0.87 pp

CALIBRATION / PREDICTED VS REALIZED

00252550507575100100perfect calibrationpredicted %realized %
View as table
bucketpredictedrealizederrorn
0-10%2.0%2.2%+0.2 pp17,506,475
10-20%14.5%14.8%+0.3 pp2,700,971
20-30%24.6%24.4%-0.2 pp1,816,738
30-40%35.1%34.8%-0.3 pp1,532,115
40-50%45.1%44.4%-0.7 pp1,303,424
50-60%55.0%54.5%-0.5 pp1,463,549
60-70%65.0%64.6%-0.3 pp1,434,872
70-80%75.3%74.9%-0.4 pp1,677,635
80-90%82.7%82.3%-0.4 pp902,153
FOXALGO / SCORECARD INSTRUMENT 04
n = 707,782,112MAE 1.39 pp

CALIBRATION / PREDICTED VS REALIZED

00252550507575100100perfect calibrationpredicted %realized %
View as table
bucketpredictedrealizederrorn
0-10%4.4%4.3%-0.1 pp291,267,133
10-20%14.4%14.6%+0.2 pp108,807,065
20-30%24.9%25.2%+0.3 pp61,440,182
30-40%34.4%34.3%-0.1 pp44,213,862
40-50%45.3%45.2%-0.1 pp38,819,526
50-60%55.1%55.2%+0.1 pp52,047,910
60-70%64.9%64.3%-0.6 pp32,951,655
70-80%75.7%75.8%+0.0 pp52,722,079
80-90%82.0%82.1%+0.1 pp25,512,700

Same buckets as the raw predicted/realized bars below, plotted against a perfect-calibration diagonal instead of read bar-by-bar. Crypto is anchored to 00:00 UTC (not 09:30 ET) and splits weekday from weekend, since it trades 24/7 with no exchange bell to anchor to.

Futures (index, energy, metals)

MAE: 0.84 pp · n = 164,920,140

predictedrealized (out-of-sample)
0-10%
predicted 3.4% · realized 3.5% · n = 56,042,860
10-20%
predicted 15.1% · realized 15.0% · n = 20,652,607
20-30%
predicted 24.8% · realized 24.8% · n = 16,450,897
30-40%
predicted 34.3% · realized 34.3% · n = 11,760,851
40-50%
predicted 45.2% · realized 45.1% · n = 12,768,405
50-60%
predicted 53.8% · realized 53.7% · n = 7,300,159
60-70%
predicted 65.9% · realized 65.8% · n = 9,908,539
70-80%
predicted 74.0% · realized 73.6% · n = 12,967,003
80-90%
predicted 86.1% · realized 86.2% · n = 16,535,975
90-100%
predicted 90.3% · realized 89.9% · n = 532,844

FX majors (mid-price)

MAE: 0.49 pp · n = 776,797,176

predictedrealized (out-of-sample)
0-10%
predicted 2.4% · realized 2.4% · n = 410,478,455
10-20%
predicted 14.7% · realized 15.2% · n = 85,032,162
20-30%
predicted 24.5% · realized 25.3% · n = 46,212,284
30-40%
predicted 35.0% · realized 35.6% · n = 48,007,249
40-50%
predicted 45.2% · realized 46.3% · n = 27,125,949
50-60%
predicted 55.3% · realized 56.2% · n = 38,443,925
60-70%
predicted 64.0% · realized 64.9% · n = 42,457,281
70-80%
predicted 75.8% · realized 76.4% · n = 36,150,469
80-90%
predicted 83.5% · realized 83.9% · n = 42,889,402
90-100%
no held-out observations in this bucket

Crypto (BTC/ETH, alts)

MAE: 1.39 pp · n = 707,782,112

predictedrealized (out-of-sample)
0-10%
predicted 4.4% · realized 4.3% · n = 291,267,133
10-20%
predicted 14.4% · realized 14.6% · n = 108,807,065
20-30%
predicted 24.9% · realized 25.2% · n = 61,440,182
30-40%
predicted 34.4% · realized 34.3% · n = 44,213,862
40-50%
predicted 45.3% · realized 45.2% · n = 38,819,526
50-60%
predicted 55.1% · realized 55.2% · n = 52,047,910
60-70%
predicted 64.9% · realized 64.3% · n = 32,951,655
70-80%
predicted 75.7% · realized 75.8% · n = 52,722,079
80-90%
predicted 82.0% · realized 82.1% · n = 25,512,700
90-100%
no held-out observations in this bucket

Buckets are ranked honestly — including the ones with the largest gap, not just the flattering ones. Weight is nheld (held-out sample size) per grid point, so thinly-sampled corners of the grid don't get equal say with the well-sampled center.

FOXALGO / SCORECARD INSTRUMENT 05
2018-01 to 2026-07worst month 15.8 pp

103 months. No resets.

JFMAMJJASOND201820192020202120222023202420252026
|gap|:0–2pp2–4pp4–7pp7–12pp12pp+
View as table
monthpredictedrealizedgapn
2026-0739.3%34.8%+4.5 pp132
2026-0639.3%40.9%+1.6 pp132
2026-0539.3%42.9%+3.5 pp126
2026-0439.3%35.7%+3.6 pp126
2026-0339.3%40.6%+1.3 pp138
2026-0239.3%38.3%+1.0 pp120
2026-0139.3%41.3%+2.0 pp126
2025-1239.3%29.4%+9.9 pp126
2025-1139.3%44.4%+5.1 pp126
2025-1039.3%35.6%+3.7 pp132
2025-0939.3%49.2%+9.9 pp132
2025-0839.3%39.7%+0.4 pp126
2025-0739.3%36.2%+3.1 pp138
2025-0639.3%50.8%+11.4 pp132
2025-0539.3%38.1%+1.2 pp126
2025-0439.3%29.4%+9.9 pp126
2025-0339.3%44.7%+5.4 pp132
2025-0239.3%38.3%+1.0 pp120
2025-0139.3%38.3%+1.0 pp128
2024-1239.3%31.8%+7.6 pp126
2024-1139.3%35.8%+3.5 pp120
2024-1039.3%41.3%+2.0 pp138
2024-0939.3%36.4%+2.9 pp132
2024-0839.3%36.5%+2.8 pp126
2024-0739.3%47.8%+8.5 pp138
2024-0639.3%44.4%+5.1 pp126
2024-0539.3%29.5%+9.8 pp132
2024-0439.3%40.9%+1.6 pp132
2024-0339.3%42.5%+3.2 pp120
2024-0239.3%39.7%+0.4 pp126
2024-0139.3%40.6%+1.3 pp138
2023-1239.3%44.7%+5.4 pp114
2023-1139.3%40.2%+0.8 pp132
2023-1039.3%40.6%+1.3 pp138
2023-0939.3%36.7%+2.6 pp120
2023-0839.3%44.2%+4.9 pp138
2023-0739.3%48.5%+9.2 pp132
2023-0639.3%49.2%+9.9 pp126
2023-0539.3%34.1%+5.3 pp138
2023-0439.3%35.0%+4.3 pp120
2023-0339.3%47.0%+7.7 pp132
2023-0239.3%34.2%+5.1 pp120
2023-0139.3%47.0%+7.7 pp132
2022-1239.3%35.8%+3.5 pp120
2022-1139.3%35.6%+3.7 pp132
2022-1039.3%42.4%+3.1 pp132
2022-0939.3%37.3%+2.0 pp126
2022-0839.3%47.8%+8.5 pp138
2022-0739.3%46.8%+7.5 pp126
2022-0639.3%39.4%+0.1 pp132
2022-0539.3%34.1%+5.3 pp138
2022-0439.3%39.5%+0.2 pp114
2022-0339.3%37.7%+1.6 pp138
2022-0239.3%40.8%+1.5 pp120
2022-0139.3%43.2%+3.9 pp132
2021-1239.3%42.1%+2.8 pp126
2021-1139.3%37.1%+2.2 pp132
2021-1039.3%41.3%+2.0 pp126
2021-0939.3%39.4%+0.1 pp132
2021-0839.3%39.1%+0.2 pp138
2021-0739.3%36.5%+2.8 pp126
2021-0639.3%34.8%+4.5 pp132
2021-0539.3%37.9%+1.4 pp132
2021-0439.3%33.3%+6.0 pp120
2021-0339.3%39.1%+0.2 pp138
2021-0239.3%42.5%+3.2 pp120
2021-0139.3%38.9%+0.4 pp126
2020-1239.3%44.4%+5.1 pp126
2020-1139.3%31.1%+8.2 pp132
2020-1039.3%39.7%+0.4 pp126
2020-0939.3%34.8%+4.5 pp132
2020-0839.3%39.4%+0.1 pp132
2020-0739.3%38.6%+0.7 pp132
2020-0639.3%37.1%+2.2 pp132
2020-0539.3%37.3%+2.0 pp126
2020-0439.3%33.3%+6.0 pp126
2020-0339.3%42.3%+3.0 pp137
2020-0239.3%48.7%+9.4 pp117
2020-0139.3%48.5%+9.2 pp132
2019-1239.3%46.0%+6.7 pp126
2019-1139.3%35.0%+4.3 pp120
2019-1039.3%42.0%+2.7 pp138
2019-0939.3%30.3%+9.0 pp132
2019-0839.3%31.8%+7.6 pp126
2019-0739.3%42.8%+3.4 pp138
2019-0639.3%38.9%+0.4 pp126
2019-0539.3%42.4%+3.1 pp132
2019-0439.3%29.4%+9.9 pp126
2019-0339.3%36.5%+2.8 pp126
2019-0239.3%40.8%+1.5 pp120
2019-0139.3%40.6%+1.3 pp138
2018-1239.3%27.6%+11.7 pp116
2018-1139.3%31.8%+7.6 pp126
2018-1039.3%40.6%+1.3 pp138
2018-0939.3%42.1%+2.8 pp126
2018-0839.3%44.7%+5.4 pp132
2018-0739.3%38.4%+0.9 pp138
2018-0639.3%44.2%+4.9 pp120
2018-0539.3%39.1%+0.2 pp138
2018-0439.3%29.5%+9.8 pp132
2018-0339.3%37.5%+1.8 pp120
2018-0239.3%35.0%+4.3 pp120
2018-0139.3%55.1%+15.8 pp138
realized touch rate (that month)predicted (fixed since 2017)
2018-012026-07

Including the months it fit worse — March 2020 is right there. Hover any bar for the exact numbers.

Month by month since 2018

Same check as the reliability tables above, but every single month instead of a pooled bucket: 09:30 ET, +0.5 ATR, on CL, ES, GC, NQ, RTY, YM. Predicted is the number fixed during the fitting period (data through 2017 only) — nothing after that touched the numbers.

overall gap
0.13 pp
|mean predicted − realized| over all 103 months
worst month
15.77 pp
2018-01 — including the months it fit worse
Brier score
0.239
over 13,278 market-sessions
20267 months · avg gap 2.49 pp · n = 900
monthnpredictedrealizedBrier
2026-0713239.3%34.8%0.230
2026-0613239.3%40.9%0.241
2026-0512639.3%42.9%0.246
2026-0412639.3%35.7%0.231
2026-0313839.3%40.6%0.241
2026-0212039.3%38.3%0.236
2026-0112639.3%41.3%0.242
202512 months · avg gap 5.21 pp · n = 1,544
monthnpredictedrealizedBrier
2025-1212639.3%29.4%0.217
2025-1112639.3%44.4%0.249
2025-1013239.3%35.6%0.231
2025-0913239.3%49.2%0.260
2025-0812639.3%39.7%0.240
2025-0713839.3%36.2%0.232
2025-0613239.3%50.8%0.263
2025-0512639.3%38.1%0.235
2025-0412639.3%29.4%0.218
2025-0313239.3%44.7%0.250
2025-0212039.3%38.3%0.236
2025-0112839.3%38.3%0.237
202412 months · avg gap 4.06 pp · n = 1,554
monthnpredictedrealizedBrier
2024-1212639.3%31.8%0.223
2024-1112039.3%35.8%0.231
2024-1013839.3%41.3%0.243
2024-0913239.3%36.4%0.233
2024-0812639.3%36.5%0.233
2024-0713839.3%47.8%0.257
2024-0612639.3%44.4%0.248
2024-0513239.3%29.5%0.218
2024-0413239.3%40.9%0.243
2024-0312039.3%42.5%0.246
2024-0212639.3%39.7%0.239
2024-0113839.3%40.6%0.241
202312 months · avg gap 5.34 pp · n = 1,542
monthnpredictedrealizedBrier
2023-1211439.3%44.7%0.249
2023-1113239.3%40.2%0.240
2023-1013839.3%40.6%0.242
2023-0912039.3%36.7%0.233
2023-0813839.3%44.2%0.249
2023-0713239.3%48.5%0.258
2023-0612639.3%49.2%0.259
2023-0513839.3%34.1%0.227
2023-0412039.3%35.0%0.229
2023-0313239.3%47.0%0.255
2023-0212039.3%34.2%0.227
2023-0113239.3%47.0%0.254
202212 months · avg gap 3.47 pp · n = 1,548
monthnpredictedrealizedBrier
2022-1212039.3%35.8%0.231
2022-1113239.3%35.6%0.231
2022-1013239.3%42.4%0.244
2022-0912639.3%37.3%0.234
2022-0813839.3%47.8%0.257
2022-0712639.3%46.8%0.253
2022-0613239.3%39.4%0.238
2022-0513839.3%34.1%0.227
2022-0411439.3%39.5%0.238
2022-0313839.3%37.7%0.235
2022-0212039.3%40.8%0.244
2022-0113239.3%43.2%0.247
202112 months · avg gap 2.08 pp · n = 1,548
monthnpredictedrealizedBrier
2021-1212639.3%42.1%0.244
2021-1113239.3%37.1%0.233
2021-1012639.3%41.3%0.243
2021-0913239.3%39.4%0.239
2021-0813839.3%39.1%0.238
2021-0712639.3%36.5%0.232
2021-0613239.3%34.8%0.229
2021-0513239.3%37.9%0.236
2021-0412039.3%33.3%0.226
2021-0313839.3%39.1%0.238
2021-0212039.3%42.5%0.245
2021-0112639.3%38.9%0.238
202012 months · avg gap 4.19 pp · n = 1,550
monthnpredictedrealizedBrier
2020-1212639.3%44.4%0.250
2020-1113239.3%31.1%0.221
2020-1012639.3%39.7%0.239
2020-0913239.3%34.8%0.229
2020-0813239.3%39.4%0.238
2020-0713239.3%38.6%0.237
2020-0613239.3%37.1%0.234
2020-0512639.3%37.3%0.234
2020-0412639.3%33.3%0.226
2020-0313739.3%42.3%0.244
2020-0211739.3%48.7%0.258
2020-0113239.3%48.5%0.257
201912 months · avg gap 4.38 pp · n = 1,548
monthnpredictedrealizedBrier
2019-1212639.3%46.0%0.253
2019-1112039.3%35.0%0.229
2019-1013839.3%42.0%0.244
2019-0913239.3%30.3%0.220
2019-0812639.3%31.8%0.223
2019-0713839.3%42.8%0.245
2019-0612639.3%38.9%0.238
2019-0513239.3%42.4%0.245
2019-0412639.3%29.4%0.218
2019-0312639.3%36.5%0.232
2019-0212039.3%40.8%0.242
2019-0113839.3%40.6%0.242
201812 months · avg gap 5.49 pp · n = 1,544
monthnpredictedrealizedBrier
2018-1211639.3%27.6%0.213
2018-1112639.3%31.8%0.222
2018-1013839.3%40.6%0.241
2018-0912639.3%42.1%0.244
2018-0813239.3%44.7%0.249
2018-0713839.3%38.4%0.237
2018-0612039.3%44.2%0.249
2018-0513839.3%39.1%0.239
2018-0413239.3%29.5%0.219
2018-0312039.3%37.5%0.235
2018-0212039.3%35.0%0.229
2018-0113839.3%55.1%0.272

Live track record

Accumulates from launch day. Every morning's map is frozen with a timestamp before the session opens (09:30 ET) and scored after the close (16:00 ET) — no cherry-picking possible. Frozen columns are never edited; only the outcome is filled in afterward.

672 frozen · 668 resolved · 4 live

FOXALGO / LIVE RECORD
APPEND ONLYFROZEN BEFORE OPEN60 in window

Live prediction record

  • 00:01 UTC (UTC-day anchor)Bitcoin-0.50 ATR (77483.88)27.6%LIVE
  • 00:01 UTC (UTC-day anchor)Bitcoin+0.50 ATR (79098.86)29.8%LIVE
  • 00:01 UTC (UTC-day anchor)Ethereum-0.50 ATR (2478.83)27.6%LIVE
  • 00:01 UTC (UTC-day anchor)Ethereum+0.50 ATR (2548.37)29.8%LIVE
  • 09:30 ETApple-0.50 ATR (308.11)39.7%✓ TOUCHED
  • 09:30 ETApple+0.50 ATR (314.49)41.8%× NOT TOUCHED · SESSION CLOSED
  • 09:30 ETWTI Crude-0.50 ATR (85.69)41.3%× NOT TOUCHED · SESSION CLOSED
  • 09:30 ETWTI Crude+0.50 ATR (87.67)39.5%× NOT TOUCHED · SESSION CLOSED
  • 09:30 ETE-mini S&P 500-0.50 ATR (7670.89)40.6%× NOT TOUCHED · SESSION CLOSED
  • 09:30 ETE-mini S&P 500+0.50 ATR (7726.11)39.7%× NOT TOUCHED · SESSION CLOSED
  • 09:30 ETGold-0.50 ATR (4611.24)39.6%× NOT TOUCHED · SESSION CLOSED
  • 09:30 ETGold+0.50 ATR (4664.36)38.4%✓ TOUCHED
  • 09:30 ETMicrosoft-0.50 ATR (475.46)39.7%× NOT TOUCHED · SESSION CLOSED
  • 09:30 ETMicrosoft+0.50 ATR (486.84)41.8%× NOT TOUCHED · SESSION CLOSED
  • 09:30 ETE-mini Nasdaq 100-0.50 ATR (29319.79)40.6%✓ TOUCHED

Frozen columns (date, symbol, level, predicted probability, price at freeze) are written once, before the session opens, and never edited — only the outcome is filled in after the close.

predictions in window
60
most recent 60, newest first
Brier score
0.239
mean squared error of predicted vs. outcome, over 56 evaluated · lower is better, 0 is perfect
calibration gap
2.75 pp
|avg predicted − realized rate| over the shown window