Recreate the unified minute-odds mart

Build polymarket_wc2026_marts.polymarket_wc2026_market_minute_odds from a clean clone or preserved raw warehouse. Complete shared setup first.

This pipeline is the minute-grain counterpart of the hourly golden mart. It reuses the match-minute path for in-game markets and adds a futures-minute path for every other registry-eligible WC2026 market over the tournament span ([2026-06-11, 2026-07-19], capped by each market's close/resolution time).

Load and validate the 104-match Scraper reference

Follow the same reference-bundle steps as Recreate the match-minute mart:

uv run make match-minute-inputs-validate

Do not continue until the command prints:

104 Scraper reference fixture rows

Create the unified minute-odds mart

Default job config refreshes the shared Polymarket event catalog and registry before minute fetch, then runs both raw minute legs. After a successful land, set the matching POLYMARKET_WC2026_MINUTE_ODDS_REFRESH_*=false flags in .env and restart Dagster to skip completed stages on reruns (see Configuration). For example, after match-minute raw is published but futures still needs work:

POLYMARKET_WC2026_MINUTE_ODDS_REFRESH_CATALOG=false
POLYMARKET_WC2026_MINUTE_ODDS_REFRESH_MATCH=false
POLYMARKET_WC2026_MINUTE_ODDS_REFRESH_FUTURES=true

uv run make minute-odds-backfill

Catalog refresh uses routine tag/series discovery (not the multi-hour slug-prefix recall audit). The job obtains:

• market inventory from the public Polymarket Gamma API;
• minute token history from the public Polymarket CLOB API;
• fixture identity and result validation from the previously activated, checksummed Scraper oddsfox.reference.v1 bundle.

It runs both raw legs:

1. Match-minute odds for the curated 104-game / 248-market inventory.
2. Futures-minute odds for registry-eligible markets that are not moneyline / soccer_team_to_advance.

Both legs share the hourly odds fetch stack: CLOB batch POST (≤20 tokens), 24-hour window pre-chunking, workers/RPS 40 with auto-tune up to 90, and a publish path that spills bounded typed Arrow batches to temporary Parquet shards, then promotes an immutable partitioned snapshot (all-token raw + primary-token minute OHLC) under the runtime root and registers stable DuckDB views (warehouse lock is not held during shard construction). Unchanged tokens reuse the prior snapshot without a CLOB refetch. Discovery includes open markets so in-tournament futures are eligible; match selection still requires closed game markets for the 104/248/496 inventory.

Then dbt builds +polymarket_wc2026_market_minute_odds_data_quality (tagged minute_odds). Raw history retains every CLOB token; primary-token minute OHLC is produced at publish and exposed as pass-through views, and the public mart is a view over that narrow fact plus market metadata:

polymarket_wc2026_marts.polymarket_wc2026_market_minute_odds
polymarket_wc2026_observability.polymarket_wc2026_market_minute_odds_data_quality

After raw legs are landed, set all three POLYMARKET_WC2026_MINUTE_ODDS_REFRESH_*=false flags and restart Dagster for a dbt-only rebuild. Measure disposable synthetic dbt timing with:

uv run make minute-odds-dbt-benchmark
# optional: MINUTE_ODDS_DBT_BENCHMARK_TIER=production-shaped

Measured on disposable hardware (candidate graph): performance tier (~10M primary / 20M raw rows) finished dbt in ~27s with no DuckDB temp spill. The prior full-backfill dbt phase on ~377M raw rows exceeded 71 minutes with ~27GB sort spill before cancel; acceptance is dbt ≤25 minutes so the measured ~34 minute raw phase keeps end-to-end under 60 minutes.

Inspect health with:

select *
from polymarket_wc2026_observability.polymarket_wc2026_market_minute_odds_data_quality;

blocking_issue_keys must be null before treating the mart as publication-ready.

Disposable live smoke (sampled)

For a cheaper end-to-end check against live Gamma/CLOB without refetching every market, run:

uv run make minute-odds-live-smoke

Cold runs reset .cache/minute_odds_live_smoke.duckdb and the disposable .cache/runtime/smoke/minute-odds-live runtime root (Parquet snapshots). The job still validates the full 104/248/496 match inventory before sampling, then fetches about 5% of match markets and 5% of futures markets independently (all tokens retained per selected market) and caps sampled futures windows to their final 24 hours. Catalog refresh uses routine tag/series discovery and does not run the multi-hour slug-prefix recall audit. dbt builds only the unified DQ select; it does not prove the full match publication gate. Warm reruns:

MINUTE_ODDS_LIVE_SMOKE_RESET=false \
MINUTE_ODDS_LIVE_SMOKE_REFRESH_CATALOG=false \
uv run make minute-odds-live-smoke

Historical API availability is not guaranteed. If Gamma or CLOB no longer returns the complete interval, use an operator's previously completed raw warehouse through the completed-warehouse route.

Troubleshooting

Failure	What to check
supply a complete operator-local 104-match schedule	Same as match-minute: exactly 104 IDs 1..104.
No registry-eligible WC2026 futures markets	Registry refresh must admit non-match markets under the volume eligibility gate.
Futures fetch empty/error	Empty in-window CLOB history is audited and skipped on publish; hard error/cancelled fail the run. Retry transient failures or rebuild from a completed raw warehouse.
blocking_issue_keys not null	Inspect the observability row and match-minute / futures audit tables.
Live smoke sample count mismatch	Confirm smoke seed/fraction defaults and that both legs recorded population_markets / selected_markets in sync_run_metrics.