Overview
Everything you see in Watchparty starts life as a message from one of two sources: OpenF1’s live feed while a session is running, or FastF1’s historical archive for backfilling past weekends. Both sources are pulled through the same pipeline, so a widget never has to know or care which one fed it.How It Works
Two Sources, One Pipeline
Live data arrives over OpenF1’s MQTT feed. This is the processed live path: topics are global rather than scoped to a session, so there’s no session discovery step, and it is materially more complete than polling OpenF1’s REST endpoints (a REST-only capture was found to collect a fraction of the lap times an MQTT capture collected in the same window). REST is kept as a raw-archive witness alongside it, not as the primary live path. Historical weekends are loaded by a backfill service that reads full seasons from FastF1 and rows them in tagged with afastf1_backfill source, through the exact same downstream path a live session takes.
Raw Before Transform
Whatever arrives, live or backfilled, is archived in its raw form before anything touches it. Only after the raw payload is safely stored does normalisation begin. If normalisation ever needs re-running, or a bug is found in it, the raw archive means nothing is lost.Single Write Path
One component, normalisation, is the only place a raw payload becomes a stored row. Live ingestion and backfill both call into it rather than writing storage rows themselves. Inside normalisation, a record missing a value field (a sector time, a gap) becomes an explicit gap rather than being thrown away; a record missing the identifying fields needed to place it at all is quarantined to a dead-letter archive instead of being silently dropped.Storage and Streaming
Normalised rows land in TimescaleDB, the platform’s time-series store. At the same moment, each normalised record is published onto a Redis channel scoped to that session and to a data type (telemetry, car location, timing, laps, race control, weather). Anything downstream, insights, replay, the gateway, watches these channels rather than polling the database.Replay Is First-Class
Any stored session can be replayed: the replay service reads it back at a chosen speed and re-publishes it onto the exact same channels a live session would use, in an envelope with the exact same shape. The only difference is asource field in the envelope saying live or replay, and every consumer is written to never branch on it. That single rule is what makes “replay any archived weekend with identical behaviour” true rather than aspirational: a widget built against a live session works against a replayed one without changes.
Session Identity
A weekend’s true identity (year, round, session type) is resolved once by normalisation and stamped onto the session as a canonical id, independent of whichever raw identifiers the source happened to use. This is what lets the picker show one card per weekend regardless of whether the underlying rows came from OpenF1 or FastF1, and prefer the canonical FastF1 data over a live capture of the same session once it exists.Where You See It in the App
- The session picker shows one card per weekend. A session still sourced live, not yet backfilled by FastF1, carries a small dot next to its label. The dot disappears on its own once FastF1 data for that session lands, and the card switches to serving that instead.
- Live vs replay, from your seat, look the same. Both stream over the same channels in the same envelope shape. Whether a given historical session is actively being replayed behind the scenes is an operational concern, not something you configure in the app.
- A connection banner tells you when a session is hydrating, when the stream drops and is reconnecting, or when a live session has ended.
FAQ
Why does a session sometimes change source after I've been watching it?
Why does a session sometimes change source after I've been watching it?
If you’re watching a session live, FastF1 has not backfilled it yet. Once it does, typically after the session ends, the picker’s card for that weekend switches to the canonical FastF1 data. The live dot disappears when that happens.
Does the webapp ever talk to Redis or the database directly?
Does the webapp ever talk to Redis or the database directly?
No. The webapp only talks to the gateway’s REST API and WebSocket stream. Everything upstream of the gateway, ingestion, normalisation, storage, streaming, insights, is invisible to it.
Next Steps
Insights and Versioning
How the metrics built on this data stream are computed and versioned.
Telemetry Degradation
What happens when the source feed itself carries bad data.
Timing Tower
See the live/replay stream driving a widget directly.
Watchparty
Back to the Watchparty overview.