> ## Documentation Index > Fetch the complete documentation index at: https://playbook.smedleygroup.com/llms.txt > Use this file to discover all available pages before exploring further. # Operator Cheat Sheet > Every Watchparty operator action as a command: launch the demo, launch live ingest, backfill from FastF1, repair from the OpenF1 raw archive, parameters, race data uploads, analytics, backups. ## Overview One page of commands for every action an operator runs against the local Watchparty stack. Each entry is a pointer, not the full story: the repository's `docs/OPERATOR_CHEAT_SHEET.md` is the source copy of this page, and `docs/LIVE_DAY_RUNBOOK.md` and `ops/local/README.md` hold the detail. All commands run from the repository root in PowerShell. > [!WARNING] > > Everything targets the DEV stack. PROD holds approved datasets only and is populated from a DEV export, never written directly. Every `ops/local` script prints a `TARGET:` line first: read it before continuing. *** ## Before Anything ### Stacks and Ports | | prod | dev | test | | - | - | - | - | | database | `vpw_prod` | `vpw_dev` | `vpw_test` (db tests only) | | Postgres / Redis | 5432 / 6379 | 5433 / 6380 | 5434 / none | | gateway / replay / web | 8000 / 8001 / 5173 | 8010 / 8011 / 5174 | | | env file | `ops/local/env/prod.env` | `ops/local/env/dev.env` | `db/docker-compose.yml` | ### Load the Environment Into a Shell Scripts under `scripts/` read `DATABASE_URL` (some read `DATABASE_DSN`) from the shell. Do this once per PowerShell window: ```powershell theme={null} .venv\Scripts\Activate.ps1 . .\ops\local\env.ps1 $v = Import-VpwEnv -Env dev -RepoRoot (Get-Location).Path $env:DATABASE_URL = $v['DATABASE_URL'] $env:DATABASE_DSN = $v['DATABASE_DSN'] $env:REDIS_URL = $v['REDIS_URL'] ``` Every script also accepts `--database-url `. The DEV DSN is `postgresql://vpw:vpw@localhost:5433/vpw_dev`. ### Stack Up, Status, Down The data volume is preserved on down. ```powershell theme={null} docker compose --env-file ops\local\env\dev.env -p vpw-dev -f ops\local\docker-compose.yml up -d --wait docker compose --env-file ops\local\env\dev.env -p vpw-dev -f ops\local\docker-compose.yml ps docker compose --env-file ops\local\env\dev.env -p vpw-dev -f ops\local\docker-compose.yml down ``` ### psql Into DEV ```powershell theme={null} docker compose --env-file ops\local\env\dev.env -p vpw-dev -f ops\local\docker-compose.yml exec timescaledb psql -U vpw -d vpw_dev ``` ### Migrate DEV by Hand The launchers migrate for you. By hand, only when you need it without launching: ```powershell theme={null} Push-Location db; ..\.venv\Scripts\alembic.exe upgrade head; Pop-Location ``` *** ## Backfill a Weekend From FastF1 Downloads one weekend's sessions from the FastF1 archive and writes them through the normal normaliser. Rows land under synthetic session keys of the form year, round, code index (FP1 00, FP2 01, FP3 02, SQ 03, S 04, Q 05, R 06) with `source = fastf1_backfill`. A live OpenF1 copy of the same session keeps its own key and is never touched; the two converge in the picker through the canonical identity map. ```powershell theme={null} python scripts\backfill_to_db.py --year 2026 --round 14 --session-codes FP1 FP2 ``` * `--session-codes` names exactly which sessions to fetch. Omit it and the script derives the full weekend from the FastF1 schedule, which fails loudly for sessions that have not happened yet. The rows that exist still land; the exit code is non-zero. * Round numbers are FastF1 rounds. 2026: Zandvoort 12, Monza 13, Madrid 14. * By default the script also precomputes each session's insight outputs, and for a race or sprint its strategy outputs. `--skip-insights` and `--skip-strategy` turn those off and leave the session data-only, with a loud analytics-incomplete warning. Compute later from the Analytics section if you skipped. * Repeat `--year N --round M` pairs to do several weekends in one run. * Cache: `data/fastf1_cache`, override with `--cache-dir`. A warm cache makes a re-download fast. * Re-running an already backfilled session is a no-op: the backfill manifest records finished endpoints and the script skips them. There is no force flag; see Refresh below. * Typical time: a few minutes per practice session on a warm network, ten or more for a race with full telemetry. > [!WARNING] > > FastF1 has no HTTP timeout. A run whose output has stopped growing for ten minutes is hung, not slow. Kill it and re-run; the manifest resumes from the last completed endpoint. ### Refresh a Weekend Use when the source data changed or a partial run went wrong. Clears the weekend's synthetic keys from every backfilled table plus its manifest rows, then backfill again. Dry run by default; live OpenF1 keys are out of scope and survive. ```powershell theme={null} python scripts\clear_session_from_db.py --year 2026 --round 14 python scripts\clear_session_from_db.py --year 2026 --round 14 --yes python scripts\clear_session_from_db.py --year 2026 --round 14 --include-analytics --yes ``` `--include-analytics` also drops the weekend's metric outputs and event parameters. Prod refuses without `--allow-prod`. ### Verify In psql: ```sql theme={null} SELECT session_key, session_name, source, date_start::date FROM sessions WHERE meeting_key = 202614 ORDER BY date_start; SELECT session_key, count(*) laps, count(DISTINCT driver_number) drivers FROM laps WHERE session_key BETWEEN 20261400 AND 20261406 GROUP BY 1 ORDER BY 1; ``` *** ## Launch the Demo The replay viewer for archived weekends. **1. One-time setup** On a cold machine, or after a data wipe. Brings up infra, installs Python dependencies into `.venv`, migrates, seeds, and backfills the two reference weekends (2025 Abu Dhabi round 24, 2026 Miami round 4) data-only. Needs Docker Desktop running, a repo-root `.venv`, and `npm install` already done in `apps/pitwall-web`. ```powershell theme={null} .\ops\local\setup-demo.ps1 -Env dev ``` **2. Every demo** Ensures infra is up and the database has sessions, then opens three windows: replay, gateway and the web server. ```powershell theme={null} .\ops\local\launch-demo.ps1 -Env dev ``` Open `http://localhost:5174`. `launch-demo.bat -Env dev` is the same thing for a double-click launch or a snapshot recipient. **3. Stop** Close the three windows. > [!NOTE] > > Replay only publishes server-computed insight channels for a session that has been precomputed (see Analytics). Sessions that have not been precomputed still replay; the webapp falls back to client-side derivations. *** ## Launch Live Ingest Full detail, including what LIVE looks like in the app and recovery steps, lives in the repository's `docs/LIVE_DAY_RUNBOOK.md`. ### The Week Before None of this needs a session row yet. ```powershell theme={null} # parameter rows for the round (CSVs live in services\parameters\data) python scripts\load_fuel_parameters.py services\parameters\data\2026_fuel_numbers.csv --year 2026 --dry-run python scripts\load_fuel_parameters.py services\parameters\data\2026_fuel_numbers.csv --year 2026 python scripts\load_strategy_parameters.py --year 2026 --rounds-csv services\parameters\data\2026_strategy_rounds.csv --overtaking-csv services\parameters\data\2026_overtaking_numbers.csv # the owner's race_data.json for the round, plus the lap-0 solve python scripts\load_race_data.py --year 2026 --round 14 --dry-run python scripts\load_race_data.py --year 2026 --round 14 # preflights in round mode (strategy + fuel rows must PASS, priors WARN is fine) python scripts\strategy_preflight.py --year 2026 --round 14 python scripts\insights_preflight.py --year 2026 --round 14 # broker credentials check: silence is success python scripts\mqtt_smoke.py ``` Optional priors from a comparable precomputed race: `python scripts\build_strategy_priors.py --source-session --year 2026 --round 14 --dry-run`, then drop `--dry-run` to write. ### Pre-Weekend Docker and WSL Guard Do this before every live weekend. Full detail is in the repository's `docs/LIVE_DAY_RUNBOOK.md`. The preflight inside `launch-live.ps1` also warns (it never blocks) on the two machine settings below, read-only. If the Store auto-update policy is absent it reads as a note rather than a warning; the `winget upgrade` check below is the interim mitigation. ```powershell theme={null} # pending WSL update check (Microsoft Store app: Library -> Settings -> App updates -> off) winget upgrade wsl --version # confirm both containers carry the restart policy once the stack is up # (the vpw-dev- prefix follows COMPOSE_PROJECT_NAME; a prod stack is vpw-prod-*) docker inspect -f '{{.HostConfig.RestartPolicy.Name}}' vpw-dev-timescaledb-1 docker inspect -f '{{.HostConfig.RestartPolicy.Name}}' vpw-dev-redis-1 ``` Docker Desktop Resource Saver: Settings (gear icon) -> General -> turn off "Enable Resource Saver" (older versions: Settings -> Resources). ### Pre-Weekend Sleep and Power The ingestion window and the insights and strategy supervisors each hold a Windows keep-awake lock while they run, which stops idle sleep and Modern Standby. The screen may still turn off. > [!WARNING] > > The keep-awake lock cannot stop a lid close or a manual Sleep. Before every live weekend: keep the lid open, stay on mains power for the whole session, and set sleep to Never on both mains and battery. The preflight inside `launch-live.ps1` warns (it never blocks) when sleep is not set to Never on mains or on battery, and when the machine is running on battery. Full detail is in the repository's `docs/LIVE_DAY_RUNBOOK.md`, "Pre-weekend sleep and power". ```powershell theme={null} # operator-run fix: sleep Never on mains and on battery (scripts never change machine settings) powercfg /change standby-timeout-ac 0 powercfg /change standby-timeout-dc 0 # while live, in an ADMIN prompt: the python.exe processes should be listed under SYSTEM powercfg /requests ``` If no `python.exe` appears under SYSTEM while live, the lock is not held (look for a `keep-awake` warning in that window's log) and the sleep settings are the only protection. ### Start Brings compose up, migrates, then opens five windows: gateway, web, insights supervisor, strategy supervisor, ingestion. OpenF1 credentials are read by key name from `$HOME\Desktop\openf1.env` (`OPENF1_USERNAME=...` and `OPENF1_PASSWORD=...`, order independent) and handed only to the ingestion window. The launcher refuses to start while an ingestion is already running or the gateway or web port is busy, and names the PID. Close the earlier windows or run `-Down` first. ```powershell theme={null} .\ops\local\launch-live.ps1 -Env dev ``` `-Year 2026` restricts the session watcher. `-VenvPath ` borrows another checkout's venv when running from a worktree. The raw archive lands in `C:\Projects\vpw-captures\live-raw\\` and strategy solve packages in `C:\Projects\vpw-captures\strategy-archive\\`. ### Logs and Heartbeat Every one of the five windows records its own output to `\logs\.log` (`gateway`, `web`, `insights-supervisor`, `strategy-supervisor`, `ingestion`), so closing a window no longer loses what it printed. Ingestion also writes `\heartbeat.log` every 60 s. Each log opens with `[start] at ` and ends with `[exit] code= at ` when the process stops. The web window alone restarts Vite if it exits: 3 s pause, logs `[restart] attempt `, gives up after 5 restarts in 10 minutes, and Ctrl+C stops it for good. When reporting a live-day problem, attach the run folder first (`C:\Projects\vpw-captures\live-raw\\`). It carries the logs and the heartbeat alongside the raw payloads, so it is a complete record of what that launch actually did. ### MQTT Client Id Defaults to `vpw-live-ingest-`, printed at launch. To use a specific id, export it before launching. It must stay stable across restarts and never be shared with another install. ```powershell theme={null} $env:OPENF1_MQTT_CLIENT_ID = 'vpw-live-ingest-' .\ops\local\launch-live.ps1 -Env dev ``` ### T-20 Minutes ```powershell theme={null} curl -H "X-API-Key: dev-key" http://localhost:8010/v1/sessions python scripts\identity_preflight.py python scripts\identity_preflight.py --restamp python scripts\strategy_preflight.py --session-key python scripts\insights_preflight.py --year 2026 --round 14 ``` Read the `solve_vs_lap` line of the strategy preflight. It compares the archived lap-0 solve wall time (`timings.total_s` in `output_manifest.json`) with the fastest driver's base-pace lap in the uploaded `race_data.json`, and warns above 60% of a lap. A WARN means live solves will lag laps and be abandoned (Singapore 2026: 72 s against laps of about 90 s at `ctl.number_repeats` 100). Lower `ctl.number_repeats` in the file (50 fixed Singapore) and re-upload with `load_race_data.py` (see Race Data Template and the Lap-0 Solve). It never fails the preflight, and `load_race_data.py` prints the same line straight after the lap-0 solve. "skipped: no lap-0 solve found" means the archive holds none for this key (pass `--archive-dir` if it lives elsewhere); "cannot check: no base pace in race\_data" means the file has no usable sector base paces. If the identity preflight reports an unresolved meeting, add the printed entry to `services/normalisation/src/vpw_normalise/reference/openf1_meeting_rounds.json`, then run `--restamp`. Without it the strategy child resolves no round parameters. The session row appears in the picker about 60 seconds after OpenF1 publishes the session. If a `race_data.json` was uploaded for this round, promote its lap-0 rows onto the live session key now, straight after the `--restamp`. See Race Data Template and the Lap-0 Solve below. > [!WARNING] > > Do not start the strategy supervisor window for a sprint. The parameter rows describe the grand prix, so a sprint solve would run on the wrong distance and fuel load. ### Stop and Restart Close each service window (Ctrl+C), then stop compose: ```powershell theme={null} .\ops\local\launch-live.ps1 -Env dev -Down ``` To restart one child mid-session, close its window and re-run just that child's block from `launch-live.ps1`, or re-run the whole script; the compose and migrate steps are idempotent and ingestion resumes from its persisted checkpoints. ### Restart the Live Strategy Child Only for a wedged child. A late or newer `strategy:` or `race_data:` row is adopted from the next lap by itself, so a restart is not needed to reload parameters. Never kill process trees by hand. ```powershell theme={null} python scripts\restart_live_strategy.py --session-key --wait-s 150 ``` It drops a flag file; the strategy supervisor notices on its next scan (up to 30 s), stops the child (its engine trees die with it), and respawns it on the scan after, not counted as a crash. The new child catches up silently and offers one lap, so there is no flood of past laps. With `--wait-s` the script follows `strategy-supervisor.log` (newest run, or `--log `) and prints the restart, respawn, warm-up and "race\_data template pinned" or "newer parameters adopted" lines. The flag folder is `%TEMP%\vpw-strategy-restart`, or the folder named by `VPW_STRATEGY_RESTART_DIR` if set (the supervisor reads the same variable). ### Post-Session Verification ```powershell theme={null} python -m vpw_ops verify --session-key ``` *** ## Backfill or Repair From OpenF1 There is no OpenF1 historical backfill. MQTT live is the only processed OpenF1 path; REST polling runs alongside as an archive-only witness. What exists is repair from the raw archive that live ingestion wrote. Every repair is a dry run until `--apply`, routes through the real normaliser mapping, and never deletes. ```powershell theme={null} # roster missing because the one-shot drivers message was missed (late join) python scripts\repair_drivers_from_raw.py --raw-dir --session-key --apply # same shape for positions, intervals (gap + laps down) and weather temperatures python scripts\repair_positions_from_raw.py --raw-dir --session-key --apply python scripts\repair_intervals_from_raw.py --raw-dir --session-key --apply python scripts\repair_weather_from_raw.py --raw-dir --session-key --apply # fill a gap from ANOTHER capture of the same session: replays a recv_at window # of an mqtt capture through the real normaliser. DEV only (refuses vpw_prod, # port 5432 and any name but vpw_dev); no Redis publish. Dry run reports NEW vs # EXIST rows per endpoint; add --apply to write, then rebuild the insights and # strategy precomputes for the session. python scripts\replay_mqtt_window.py --capture-dir --session-key --from --to --database-url postgresql://vpw:vpw@localhost:5433/vpw_dev # live rehearsal: feed an archived mqtt capture (laps, stints, pit, race_control, # drivers) paced under the scratch twin key 20269906 (2026 round 99), so a live # strategy supervisor sees laps arrive. DEV only (vpw_dev off 5432; Redis must be # given and not 6379; never PROD). Dry run is the default and connects to nothing. # Order: --prepare --apply, load_race_data.py --year 2026 --round 99 , # strategy_preflight, feeder (pre-feed, then a 120 s countdown), start the # supervisor in the countdown with NO VPW_CLOCK_OFFSET_S. Details in the script docstring. python scripts\rehearse_mqtt_feed.py --capture-dir --source-session-key --database-url postgresql://vpw:vpw@localhost:5433/vpw_dev --redis-url --prepare --apply python scripts\rehearse_mqtt_feed.py --capture-dir --source-session-key --database-url postgresql://vpw:vpw@localhost:5433/vpw_dev --redis-url --raw-archive-dir --prefeed-lap 10 --speed 6 --apply # afterwards print the DEV cleanup statements (executes nothing) python scripts\rehearse_mqtt_feed.py --cleanup-sql # split absorbed laps (lap N carrying lap N+1's times) python scripts\repair_laps_from_raw.py --session-key --archive-root --apply # canonical identity re-stamp, including a wrongly stamped round python scripts\repair_session_identity.py python scripts\repair_session_identity.py --apply # sessions.gmt_offset (circuit-local UTC offset). --source live (default) # repairs source='live' rows from the OpenF1 sessions endpoint; --source # fastf1 repairs source='fastf1_backfill' rows from the FastF1 event # schedule (FastF1's GmtOffset is treated as a verified circuit-local offset). # Dry run first, DEV DSN shown; --allow-prod is refused by default for either mode. python scripts\repair_gmt_offset.py --database-url postgresql://vpw:vpw@localhost:5433/vpw_dev python scripts\repair_gmt_offset.py --apply --database-url postgresql://vpw:vpw@localhost:5433/vpw_dev python scripts\repair_gmt_offset.py --source fastf1 --database-url postgresql://vpw:vpw@localhost:5433/vpw_dev python scripts\repair_gmt_offset.py --source fastf1 --apply --database-url postgresql://vpw:vpw@localhost:5433/vpw_dev ``` `` is the `raw` tree under the live-raw stamp directory for that day. `--raw-dir` and `--session-key` can repeat. The `gmt_offset` repair is the one entry here that needs no raw archive: it looks each session's circuit-local UTC offset up by that session's own key, from OpenF1 for live rows or from the FastF1 event schedule for backfilled rows, and writes that single column. A session the lookup cannot resolve is reported and left NULL, never guessed. Every clock in the webapp reads track-local time from this column, falling back to browser-local time only when it is absent. ### Raw-Only Capture Records OpenF1 REST to disk for later analysis and writes nothing to the database: ```powershell theme={null} python scripts\capture_live_openf1.py --smoke python scripts\capture_live_openf1.py --out ``` Capture rehearsal through `launch-live.ps1 -Capture` is retired: the capture server is HTTP-only and ingestion now requires MQTT. *** ## Parameters Parameters are versioned rows, never constants. Loads are append-only: identical values are a no-op, changed values write a new active version. ```powershell theme={null} python scripts\load_fuel_parameters.py services\parameters\data\_fuel_numbers.csv --year [--dry-run] python scripts\load_strategy_parameters.py --year --rounds-csv services\parameters\data\_strategy_rounds.csv --overtaking-csv services\parameters\data\_overtaking_numbers.csv [--dry-run] python scripts\load_default_parameters.py --evo-s-per-lap --cutoff-pct [--dry-run] python scripts\load_circuit_parameters.py --circuit-key --evo-s-per-lap [--cutoff-pct ] [--dry-run] python scripts\build_strategy_priors.py --source-session --year --round [--dry-run] ``` Reload the fuel row after Saturday qualifying once the real Q time is in the CSV. Stored insights computed on the old row are stale until recomputed. `load_default_parameters.py` changes the value for every track; `load_circuit_parameters.py` changes it for one, and wins over the global value at that circuit. Precedence runs defaults, then circuit, then round, then session. Find a circuit key with `SELECT DISTINCT circuit_key, country_name FROM sessions WHERE year = `. Pass every pin you want that circuit to keep in the same command: a key you do not pass keeps the circuit's existing pin if it has one, otherwise it is filled from the global row. `--deactivate` drops the circuit's pin so the global value applies again. > [!NOTE] > > Neither command rewrites rows that are already stored. A parameter change takes effect at the next precompute or live run, so re-precompute the sessions you care about (see Analytics After Data Lands). `race_data:{year}:{round:02d}` is a parameter family too, but it is loaded from the owner's uploaded file rather than a CSV, and it carries a lap-0 solve with it. It has its own section below. *** ## Race Data Template and the Lap-0 Solve The owner's `race_data.json` for one round. Uploading it does four things in a single command: validates the file as a template, writes `race_data:{year}:{round:02d}` to `event_parameters` (append-only, like every other parameter family), creates a provisional race session `YYYYRR90` if the real one does not exist yet, and solves lap 0 into the strategy archive so a pre-race prediction is stored before a wheel turns. Keep uploaded files at `C:\Projects\vpw-captures\race-data-uploads\\-\race_data.json`, with a `sha256.txt` beside each. The parameter row carries the whole uploaded file plus `race_data_source = {filename, sha256}`, so a file on disk can always be matched to the version that ran. > [!WARNING] > > The `race_data.json` written into the solve archive is a derived file: gaps are re-based to the field minimum and a `platform` block is added. It is not the uploaded file, so the archive is not a backup of the original. The complete copy of the upload lives in the `event_parameters` row and in whatever you keep on disk. `ctl.num_strategies` is optional (whole number, at least 1, default 5) and sets how many ranked strategy categories the engine carries per driver. ### Validate First No database connection and no writes. ```powershell theme={null} python scripts\load_race_data.py --year 2026 --round 14 --dry-run C:\Projects\vpw-captures\race-data-uploads\2026\14-madrid\race_data.json ``` It refuses on any problem and lists all of them at once. Warnings print and do not block. ### Upload and Solve Lap 0 Windows only, because the lap-0 solve runs the engine bundle. ```powershell theme={null} python scripts\load_race_data.py --year 2026 --round 14 C:\Projects\vpw-captures\race-data-uploads\2026\14-madrid\race_data.json ``` Straight after the lap-0 solve the script prints one `OK:` or `WARN:` line comparing the solve time with the base-pace lap (60% limit), naming `ctl.number_repeats` as the lever when it warns. It never changes the exit code. A running live strategy child picks up a late `strategy:` or `race_data:` load, or a newer version of one, from the next lap it processes (one INFO log line names the old and new versions); no restart is needed. If a driver in the file will not start (late driver change), re-upload the file without them before the race: a template driver who never posts a lap holds the opening solves until lap 3. `--race-date ` sets the provisional session's timestamp, defaulting to qualifying plus one day. `--archive-dir`, or the `VPW_STRATEGY_ARCHIVE_DIR` environment variable, moves the solve archive off the default `.\strategy-archive`. Re-running clears this service's own `strategy_*` rows at trigger lap 0 first, so a corrected file can be re-uploaded safely, and the parameter row gains a new version rather than overwriting the old one. Confirm the row landed with `python scripts\strategy_preflight.py --year 2026 --round 14`. The `race_data_row` check is a hard failure in round mode. These lap-0 rows survive a later `precompute_strategy.py` run, so the upload and the precompute can be done in either order. ### Associate the Lap-0 Rows With Every Race Key The lap-0 solve runs once per round, never once per session key. At upload time `load_race_data.py` copies its own lap-0 rows onto every other non-provisional race session key of the round, so opening either the OpenF1 live key or a FastF1 backfill twin in the picker shows lap 0. A backfill twin that only appears after the upload, which is the usual case because FastF1 keys land after the race weekend, has nothing to copy from at upload time; `precompute_strategy.py` picks this up for its own `--session-key` before it replays, so running the precompute against the twin associates it. If neither has run for a given twin yet, repair it directly. ```powershell theme={null} python scripts\associate_lap0_rows.py --year 2026 --round 14 --dry-run python scripts\associate_lap0_rows.py --year 2026 --round 14 ``` Copies only, never a second solve and never a delete, so it is always safe to re-run. ### Promote at T-20 Once OpenF1 has published the real race session and `identity_preflight.py --restamp` has stamped it, move the lap-0 rows off the provisional key onto the live one and remove the provisional session. Both happen in one transaction. ```powershell theme={null} python scripts\promote_provisional_race.py --year 2026 --round 14 --session-key --dry-run python scripts\promote_provisional_race.py --year 2026 --round 14 --session-key ``` It refuses unless the live key resolves to (year, round, R) through a non-provisional identity map row. It also renames the archive directory from the provisional key to the live one, but only within the root it is given, so pass the same `--archive-dir` the upload used. Running `python scripts\strategy_preflight.py --session-key ` warns if a stale provisional session is still present for the round. *** ## Analytics After Data Lands ```powershell theme={null} # insight metric outputs for one session (clears that session's rows first) python scripts\precompute_insights.py --session-key # strategy outputs for a race or sprint session; Windows only python scripts\precompute_strategy.py --session-key # re-precompute sessions whose manual lap exclusions changed python scripts\reconcile_exclusions.py # one pass, then exit python scripts\reconcile_exclusions.py --watch # keep polling every 15 s # just one session (repeat the flag to name several) python scripts\reconcile_exclusions.py --session-key ``` A pass with no `--session-key` covers every stale session, which on a fresh environment means every session anyone has ever excluded a lap on, because the reconcile bookkeeping table starts empty. Name the session when only one is meant. > [!NOTE] > > `--session-key` narrows a pass, it never forces one. A named session that is not stale is reported and left alone, and a named session whose newest lap is under an hour old is deferred exactly as it would be in a full pass, because clearing its rows under a live engine would leave a permanent hole in the timeline. To force a recompute regardless of staleness, use `precompute_insights.py --session-key `. Re-run a precompute after any model version bump for a metric it covers, otherwise replay skips that metric rather than publish two versions. `--keep-existing` skips the clear step and is only safe when no version changed. Precompute only the sessions being looked at; no historical sweeps during active development. To time the real precompute path or compare its output against what is already stored, with zero database writes (`metric_outputs` untouched, no rows cleared): ```powershell theme={null} python scripts\precompute_insights.py --session-key --output-file out.jsonl ``` Rows land in `out.jsonl` as one canonical JSON line each, in the `metric_outputs` column shape, and the run ends with a per-metric row-count table. `--output-file` and `--keep-existing` cannot be combined. A full race precompute takes about five minutes rather than 100. ### What-If Studies on the Strategy Side To try a different race-data file on a stored race, or just a timing and comparison run, with zero database writes (no clear, no lap-0 association, no `metric_outputs` write at all): ```powershell theme={null} python scripts\precompute_strategy.py --session-key 11369 --race-data C:\Projects\vpw-captures\race-data-uploads\2026\14-madrid\race_data_whatif.json ``` The session's real stored laps still drive the replay (the database is read for those, and for any manual stint-compound corrections); only the engine's own per-lap input changes to the given file instead of the database's resolved parameters. Outputs land under the `studies\` folder inside `--archive-dir` (default `.\strategy-archive`, or `VPW_STRATEGY_ARCHIVE_DIR`): the collected rows as JSON Lines at `studies\.jsonl` and the per-lap packages at `studies\\\lap_NNN\`. The stem encodes the session key, the file's own name, the first 12 hex characters of its sha256 and the run's UTC start time, so two runs more than a second apart never collide. Both paths print at the end. Nothing prunes these study folders; delete them once a study is finished. `--output-file PATH` overrides only the JSONL path, and an existing non-empty file at that path is refused rather than overwritten. `--output-file` alone (no `--race-data`) is also a no-write study: the engine's per-lap input stays the database's own resolved parameters, only the output moves to a file. Neither study flag can be combined with `--keep-existing` or `--clear-all` (exit 2): those two only mean something for a database write, which a study never makes. A study replays from lap 1 and never produces the lap-0 pre-race row, so its row count is two short of a stored session's when compared row for row. > [!NOTE] > > The lap-0 pre-race rows from an uploaded `race_data.json` survive a strategy re-precompute: the clear spares `trigger='pre_race'`. They are removed anyway, with a warning, if they carry a different model version from the run, because a metric holding two versions is skipped by replay altogether; rebuild them with `load_race_data.py` when that happens. `--clear-all` deletes them deliberately. *** ## Check a Race Against the FIA Timing PDFs The FIA publish two timing documents per race: the Race Lap Analysis and the Race History Chart. Between them they give every car's lap time and the clock time at which it completed lap 1, so comparing them against stored rows is the quickest proof that a weekend ingested cleanly. Read-only against the database. Keep the PDFs with the round's other uploads in `C:\Projects\vpw-captures\race-data-uploads\\-\`, alongside `race_data.json` and `sha256.txt`. PDFs dropped at the repository root are gitignored so they cannot be committed by accident. ```powershell theme={null} $env:DATABASE_URL = "postgresql://vpw:vpw@localhost:5433/vpw_dev" $up = "C:\Projects\vpw-captures\race-data-uploads\2026\14-madrid" # lap times per driver, the lap-1 table, the lap-1 start check, and # cumulative gap-to-leader at sample laps python scripts\compare_fia_lap_analysis.py ` --pdf "$up\2026_14_esp_f1_r0_timing_racelapanalysis_v01.pdf" ` --history-pdf "$up\2026_14_esp_f1_r0_timing_racehistorychart_v01.pdf" ` --session-key [--csv out.csv] # gap-to-leader per lap against the same history chart, by regime python scripts\compare_fia_gap_to_leader.py ` --history-pdf "$up\2026_14_esp_f1_r0_timing_racehistorychart_v01.pdf" ` --session-key [--csv out.csv] ``` Laps 2 onwards should be exact, to the millisecond. Madrid 2026 matched 1083 of 1083. Anything else is a real ingestion gap worth chasing. Lap 1 always differs, by roughly +0.3 s to +0.5 s, a different amount per car. This is upstream and expected: OpenF1 times lap 1 from the shared lights-out instant, the FIA from each car's own start-line transponder trigger. The start check confirms which of the two is at fault and prints a verdict line. At Madrid it pinned the stored race start to within 61 ms, so there was nothing to fix on our side. > [!NOTE] > > The race history chart is indexed by the leader's lap, not by each car's own lap. A lapped car simply has no row for the lap it lost, so its later rows sit one index higher than its own lap count. The scripts handle this; a hand comparison that ignores it produces a few hundred false differences. *** ## New Circuit Needs real car location rows from a session at the venue. Read-only against the database; it prints a paste-ready registry entry. Two cases: **Circuit already raced in a backfilled season** (the normal case): fit it before the weekend from the previous year's Race session, which has 300k to 460k location rows. Race sessions do not need `--laps-only`. Paste the entry into `apps/pitwall-web/src/circuits/registry.ts` keyed by the verbatim backfill circuit name only. Then, once the first live OpenF1 row of the weekend has landed (FP1), add the one-line circuit key entry pointing at the same constant, quoting the observed session key. Never invent the key beforehand. In the same pass, run the identity preflight from the T-20 steps (`identity_preflight.py`, then `--restamp`): a new weekend's OpenF1 meeting key is not in the round map until you add it, and until the live row is stamped the picker shows the live session and its FastF1 backfill as two separate rows instead of one slot with two badges (Baku 2026 FP1). ```powershell theme={null} python scripts\fit_circuit_calibration.py --session-key --circuit-id ``` **Brand-new venue** (Madrid 2026 was the first): nothing to fit until FP1 has run, so this happens mid-weekend. Practice data needs `--laps-only` or garage and pit-lane points wreck the fit. ```powershell theme={null} python scripts\fit_circuit_calibration.py --session-key --circuit-id --laps-only ``` A sane fit has a median residual under about 5 m and an extent ratio near 1.0; anything else is left out of the registry (the widget degrades explicitly) rather than committed. Geojson ids are the filenames under `apps/pitwall-web/public/circuits/circuits/`. ### Measure the Start Offset for a Track Definition `startOffset` is the fraction `[0, 1)` of the vendored geojson outline's own arc length at which the real start/finish line sits, so the Track Dominance widget can rotate the painted elements to line up with it (the outline's first point is not necessarily the line). Measure it from the previous year's Race session on DEV: take the race winner (the driver with the most completed laps), read laps 10 to 14, and take the median of the five per-lap fractions. Read-only against DEV. `--calibration` takes the circuit's committed registry constants as `scale,rotationRad,tx,ty[,mirrored]` and `--geojson` its outline file. The printed "median fraction" is what goes into the definition's `startOffset`. ```powershell theme={null} .venv\Scripts\python.exe scripts\measure_start_offset.py --session-key 20251706 --driver 1 --laps 10 11 12 13 14 --geojson apps\pitwall-web\public\circuits\circuits\az-2016.geojson --calibration "0.099949,0.013903,833.99,419.23,false" --database-url postgresql://vpw:vpw@localhost:5433/vpw_dev ``` The worked example is Baku from the 2025 race. Five of five laps should measure; a spread of more than about 0.005 between the per-lap fractions means the calibration or the lap starts need a second look before the median is committed. ### Measure Sector Lines for a Track Definition The Telemetry and Track Dominance widgets pin every lap's own S1 and S2 positions (its integrated distance at the official cumulative sector times) to a circuit's canonical `sectorLinesM` in `apps/pitwall-web/src/circuits/definitions/*.json`. The lines are not published anywhere, so they are measured from stored laps: the median length-scaled S1 and S2 distance across every timed lap with all three sector times. Read-only against DEV. ```powershell theme={null} .venv\Scripts\python.exe scripts\measure_sector_lines.py --session-key 11365 --session-key 11369 --track-length-m 5474 --database-url postgresql://vpw:vpw@localhost:5433/vpw_dev ``` Pass several `--session-key` values to pool laps across sessions of the same circuit; `--track-length-m` comes from the definition's own `trackLengthM`, never assumed. The printed `s1_m` and `s2_m` are what goes into `sectorLinesM`; the printed p10 and p90 spread for each line is worth checking before committing a wide band. A band of tens of metres or more means degraded telemetry, not a genuinely wide sector line (2026 Budapest is the known example: re-run against a healthy season's sessions rather than commit it). The session keys used for each committed circuit are recorded in that circuit's `sectorLinesM` commit message. *** ## Backups and Promotion ```powershell theme={null} # manual snapshot of a stack's volume (archive + manifest) .\ops\local\export-db.ps1 -Env dev .\ops\local\export-db.ps1 -Env prod -OutDir D:\handover -Fast # nightly export with retention (preview, then real) .\ops\local\export-db-nightly.ps1 -Env dev -DryRun .\ops\local\export-db-nightly.ps1 -Env prod # restore an archive into a stack; -Force REPLACES that stack's whole volume .\ops\local\setup-from-dump.ps1 -Env dev -Archive .\ops\local\setup-from-dump.ps1 -Env dev -Force -Archive ``` Promotion of an approved dataset is an export from DEV restored into PROD through the same two scripts. The restore prints `Every table matches the manifest (...)` when every count matches. *** ## Gates Before Committing or Merging ```powershell theme={null} python run_all_offline.py --fast # per commit, about 30 s python run_all_offline.py # before merging to main ``` The full gate replays one random real golden fixture; pin it with `$env:VPW_GOLDEN_SESSION = 11353` while other fixtures are stale. *** ## Default Webapp Layout The webapp's default workspace (tabs, open widgets, window geometry) is the owner's own captured layout, held verbatim in `apps/pitwall-web/src/store/defaultWorkspace.json`. Re-capturing it is a paste, not a code change. **1. Copy the layout out of the browser** On the Watchparty tab you actually use, press F12, open the Console, and run the line below. Make the tab you want the workbook to open on the active one first, because the capture records it. ```js theme={null} copy(localStorage.getItem("pitwall.workspace.v1")) ``` **2. Paste it over the JSON file** Replace the whole contents of `defaultWorkspace.json` with the clipboard. Pretty-printing is fine; the values must stay exactly as captured, so never tidy the file by hand. **3. Run the webapp tests and commit** The tests prove the default equals the pasted blob as loaded, then the usual gate applies. ```powershell theme={null} cd apps\pitwall-web npx vitest run ``` > [!NOTE] > > A browser that already holds a stored workspace keeps it. The new default only shows on a first-ever load and after **Reset layout** in the transport footer. Each origin (port) has its own storage, so capture from the build you actually use. *** *** ## Next Steps > **[Data Journey](/software/internal/watchparty/concepts/data-journey)** > > How live and backfilled data travel the same pipeline. > **[Insights and Versioning](/software/internal/watchparty/concepts/insights-and-versioning)** > > Why a precompute must be re-run after a model version bump. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.