The full modeling story — with the losing numbers too
How It Works

1 · The pipeline at a glance

Every week the system ingests data from five sources (DataGolf, the PGA Tour's GraphQL API, DraftKings, Open-Meteo weather, and a fantasy league site) through ~30 scraper scripts into a DuckDB database — 1.4M+ tournament-stat rows covering 2016–2026. From that history it computes 62 features per player, runs them through calibrated Random Forest classifiers to produce win / top-5 / top-10 / top-20 probabilities, cross-checks them with a 10,000-run Monte Carlo simulation, converts sportsbook odds into fair probabilities to find mispriced markets, and publishes everything here. The whole cycle — ingestion, prediction, settlement, grading, recalibration, deployment — runs automatically on a weekly schedule.

2 · The model: 48,000 rows of history, one question

The training table has 48,592 rows — one per player per tournament since 2016. Each row holds the player's stats as they stood before that event(strokes-gained form, world rank, course history) plus what actually happened: did they win, make the top 5, top 10, top 20. The model's only job: given the before-stats, predict the probability of the after-outcome. Winning is rare — 0.77% of rows — and that imbalance drives most design choices below.

The classifiers are Random Forests: hundreds of decision trees, each trained on a random resample of the data and deliberately constrained — max_depth=5, min_samples_leaf=25. Those constraints stop any tree from memorizing individuals. Without them, a tree happily learns "rank 240–260 and exactly 12 events played → 100% win rate" because one lucky qualifier fits that description once in history. Requiring 25+ players per leaf forces every probability to come from a real sample. A prediction like "5.5% to win" means precisely: of past player-weeks that resembled this profile, about 1 in 18 won.

3 · Validation: time only moves forward

Golf data is a time series, so the cardinal sin is letting the model peek at the future. Training uses only years before 2025; nothing after is touched during tuning. And because every tuning decision was made by looking at 2025 results, the 2025 score is slightly flattered — like grading yourself on the practice exam you studied from. The 2026 season was never used for any decision, making it the honest number:

MarketAUC 2025 (tuning year)AUC 2026 (untouched)
Win0.8370.809
Top 50.7500.769
Top 100.7330.732
Top 200.7360.705

AUC measures ranking skill: 0.81 for the win model means that given a random winner and non-winner, the model rates the winner higher 81% of the time (0.5 = coin flip).

4 · Calibration: does 10% mean 10%?

A model can rank players well and still exaggerate — saying 30% when reality is 18%. For betting, that's fatal: expected value is computed from the probability itself, not the ranking. So each forest is wrapped in isotonic calibration (CalibratedClassifierCV), which learns a monotonic correction from predicted to observed frequencies on held-out folds. The proof is in a full season of graded predictions — 3,171 across 29 tournaments:

MarketPredicted avgActual rateRatio
Win0.97%0.95%0.98
Top 54.89%5.39%1.10
Top 109.78%10.47%1.07
Top 2018.32%21.04%1.15

A ratio of 1.00 is perfect; the season ran 0.98–1.15. The published probabilities also blend market odds (capped at 25% weight for top-ranked players) and expert consensus (12%) — so part of that calibration comes from the market itself, not pure model skill. That distinction matters in the benchmark below.

5 · Monte Carlo: playing the tournament 10,000 times

Point probabilities can't answer questions like "how often do these three players all cash?" — for that you need whole simulated tournaments. Each player gets a scoring distribution: a mean from model-projected strokes gained and a standard deviation estimated from their own round-to-round variance (minimum 20 rounds). The simulator then plays each round as a random draw, applies a small 0.06 autocorrelation between rounds (hot streaks are real but weak — a DataGolf research finding), cuts the field to 65 after round two, and tallies finishing positions across 10,000 tournaments. The counts become probabilities, and the full position matrix feeds lineup projections. Sampling uses the Gumbel-max trick, which turns sequential "draw without replacement" into one vectorized numpy operation — all 10,000 simulations compute simultaneously, and the simulated rates match the model's probabilities to within 0.003 MAE.

6 · Odds, vig, and what "edge" really means

Sportsbook odds are not probabilities — they include the book's margin ("vig"). Convert every player's top-10 odds into implied probabilities and they'll sum to ~12 when only 10 spots exist; the extra ~20% is the book's cut. De-vigging removes it: for pool markets each book's implied probabilities are proportionally rescaled to sum to the number of paid spots — always per book, never pooled across books (mixing books halves the apparent vig and manufactures phantom edges; that was a real bug, found and fixed). Binary markets like make-cut get a flat ~5% correction. An "edge" is then model probability minus fair probability, with a 1.5-point minimum before anything is recommended, staked by half-Kelly.

The 2026 results are the best lesson in the whole project — 776 graded bets:

MarketRecordROI
Make cut10/10+52.9%
Head-to-head, round 425/58+4.1%
Head-to-head, rounds 1–380/239−24% to −41%
Group / placement / outright56/463−8% to −100%
Total174/776−20.9%

A well-calibrated model lost 22% — because being right about probabilities is not the same as beating the market's probabilities by more than the vig. Where the system won tells the story: make-cut markets (books price them laziest) and round-4 matchups (where the live data pipeline updates faster than the lines). Those two niches are the 2027 focus; the broad markets go to paper-trading.

7 · Benchmark: head-to-head with the industry standard

DataGolf is the reference model in golf analytics. Their pre-tournament predictions were snapshotted beforeeach event all season — no hindsight — giving a fair head-to-head on 1,492 matched predictions across 14 tournaments. Result: DataGolf wins on log loss by 4–5% in every market (win: 0.0443 vs 0.0463) and is clearly better at fine-grained ranking (Spearman 0.378 vs 0.310). This system is better calibrated in all four markets (win ratio 1.017 vs 1.040) — though, honestly noted, part of that calibration edge comes from blending market odds, which DataGolf's pure-model numbers don't do. For a solo project against a decade-old commercial model, within 5% on log loss is the result I'm proudest of on this page.

8 · Known limitations

Things I would tell a reviewer before they found them:
· Validation has been upgraded from a single temporal split to walk-forward CV (train on all years before Y, test on Y, for seven seasons) — the tables above show single-split numbers; the walk-forward view adds a ±0.03–0.08 year-to-year spread and revealed a real multi-season decline in predictability that DataGolf's model shares (rising parity).
· The 0.20 cap on win probability is a heuristic patch for over-confident favorites, not a modeled fix.
· Small no-cut fields (playoffs, signature events) use a model trained mostly on 144-player cut events, and the simulator hard-codes a 65-player cut.
· Closing-line value was never actually measured in 2026: an audit found the recorded "+12.2pt average CLV" compared bet prices against in-play odds on a winner-biased 6% sample. True closing snapshots are now captured pre-R1 for 2027. A 15-week bet-logging gap was likewise found and recovered from per-tournament files — the P&L above includes it.
· Player-name matching across five data sources is fuzzy by nature; normalizers handle most of it, manual mappings catch the rest.
Built solo as a learning project — Python, scikit-learn, XGBoost, DuckDB, FastAPI, Next.js. Every number on this page is reproducible from the repository: model metrics from outputs/model_test_metrics.json, benchmark from outputs/benchmark_vs_dg_summary.json, season results from outputs/season_2026_retrospective.md.