> ## Documentation Index > Fetch the complete documentation index at: https://playbook.smedleygroup.com/llms.txt > Use this file to discover all available pages before exploring further. # Deployment And The Pipeline > How a merge becomes a running change: the CDK pipeline, the two stages, the images it pushes to ECR, the distribution it publishes to S3, and the AWS resources it creates. Nothing in Telemetry is deployed by hand, and nothing is built on a track machine. One CDK pipeline per stage builds the container images, publishes the `distribution/` folder, and deploys the AWS side. The track machines then pull, on their own schedule. This page is how a change ships. For what the deployed cloud resources then do with a run, see [DAQ Upload and Conversion](/software/developer/estate/systems/telemetry/daq-pipeline). ## Two Stages, Two Pipelines The CDK app defines one stack per stage, `TelemetryDev` and `TelemetryProd`, and each creates its own self mutating `CodePipeline` named `TelemetryPipeline-{stage}`. > [!NOTE] > > **The stage is the branch.** The pipeline source is the GitHub repository connected through CodeConnections, and the branch it watches is the stage name itself. So `dev` builds from the `dev` branch and `prod` builds from the `prod` branch. Merging to `prod` is the act of releasing. Everything the pipelines create is suffixed with the stage, so the two are fully separate: separate ECR repositories, separate queues, separate Lambdas, and separate buckets. Both deploy into the same AWS account and region, `eu-west-2`. > [!WARNING] > > The pipeline is `selfMutation: true`, so it updates its own definition from the branch before running the rest. A change to the pipeline itself takes effect on the run after the one that introduced it. ## What A Run Does A run has three parts, in order. **1. Synth** Builds the Python Lambda bundles as a side input, then installs and synthesises the CDK app. The Lambda step installs each function's `requirements.txt` into its own folder and zips it, and the synth step copies those zips into `build_assets/` before running `cdk synth`. **2. ECR repositories, then the images** A stage creates the ECR repositories, and eight docker build steps then run after it, one per image. Each logs in to ECR, builds from its own context and Dockerfile, and pushes. **3. The application stack** Deploys the topic, queues, Lambdas and bucket notifications that make up the cloud side of the pipeline. The distribution publish also runs after the ECR stage, alongside the image builds. ## The Images Eight images are built on every run, each pushed to `telemetry/{image}-{stage}` in ECR: | Image | Built from | | - | - | | `receiver` | `Container.FromTcp.Consumer.ToRaw.Producer/Dockerfile` | | `raw-to-telemetry` | `Container.FromRaw.Consumer.ToTelemetry.Producer/Dockerfile` | | `telemetry-to-influx` | `Container.FromTelemetry.Consumer.ToInflux.Leaf/Dockerfile` | | `raw-to-file` | `Container.FromRaw.Consumer.ToFile.Leaf/Dockerfile` | | `exporter-api` | `telemetry/telemetry-api/Dockerfile` | | `offloaded-data-converter` | `telemetry/telemetry-api/src/Dockerfile` | | `telemetry-ui` | `telemetry/telemetry-web-app/Dockerfile` | | `telemetry-cron` | `telemetry/telemetry-cron/Dockerfile` | The first seven run on a track machine. `offloaded-data-converter` is the odd one out: it is built from the same API project but deployed as the conversion Lambda in AWS, not to a track. ### Tags Each image is pushed **twice**, as `latest` and as the resolved commit SHA of the run that built it. > [!NOTE] > > That SHA tag is the only version handle in the system. The track scripts pull `latest`, so the SHA is what lets you identify exactly what a track is running, and what you would pin to in order to go back. > [!WARNING] > > Each repository has a lifecycle rule keeping only the **last 3 images**. Rollback is therefore limited to roughly the last three merges to that branch, and older SHAs are deleted rather than archived. Tags are mutable, so `latest` moves under you. ## The Distribution Publish The step that actually reaches the tracks is a one line sync: ``` aws s3 sync distribution/ s3://telemetry-data-sg-{stage}/distribution/ --delete ``` > [!WARNING] > > `--delete` means the bucket is made to match the branch exactly. A file removed from `distribution/` in the repository is removed from the bucket on the next run, and the next track machine to run setup loses it. There is no versioning on this prefix. This is the first half of getting a change to a track. The second half is a person running `setup_application.bat` at that track, which is normally once per event. See [Getting An Update Onto The Machine](/software/developer/estate/systems/telemetry/track-operations#getting-an-update-onto-the-machine). > [!NOTE] > > So a merge to `prod` never changes a track by itself. It changes what a track will pick up the next time someone runs setup there, which is why the same change can be live at one region and not another for weeks. ## What The Application Stack Creates The cloud side is small: one topic, three queues and two Lambdas. | Resource | Name | Role | | - | - | - | | SNS topic | `TelemetryRawOffloadedDataIngestTopic-{stage}` | Receives the bucket notification for a new DAQ upload, and is also the hook a reconversion is published to | | Bucket notification | On `telemetry-data-sg-{stage}` | Fires on object created under the `daq-data/` prefix with a `.zip` suffix, into that topic | | SQS queue | `TelemetryRawOffloadedDataIngestQueue-{stage}` | Subscribed to the topic. Triggers the converter | | SQS queue | `TelemetryRawOffloadedDataIngestDLQ-{stage}` | Dead letters from the above | | SQS queue | `TelemetryReconversionNotificationQueue-{stage}` | Subscribed to TalentID's event imported topic | | Lambda | `TelemetryRawOffloadedDataConverterLambda-{stage}` | The converter. A container image function, from the `offloaded-data-converter` repository | | Lambda | `TalentIdToTelemetryReconversionChecker-{stage}` | Python. Decides what needs reconverting after a timing import | | Secret | `GklPlatformApiKeySecret-{stage}` | The GKL Platform API key the converter uses | Both Lambdas run with **3008 MB** and a **5 minute** timeout. > [!WARNING] > > The ingest queue has `maxReceiveCount: 1`, so a message that fails once goes straight to the dead letter queue with no retry. The DLQ keeps it for 5 days and the main queue keeps messages for 1 day. A conversion that fails is therefore a dead letter within minutes and gone within five days, which is the real reason a failed conversion needs a deliberate re-upload rather than waiting. See [Triggering A Reconversion Deliberately](/software/developer/estate/systems/telemetry/timing-and-lap-splitting#reconversion). > [!NOTE] > > The converter Lambda's image tag is resolved at synth time from the build's commit SHA, falling back to `latest`. So the cloud converter is pinned to the exact image from its own pipeline run, unlike the track machines which always pull `latest`. ## Deploying By Hand The pipelines are the supported route. The CDK app is a standard TypeScript project, so `npx cdk synth` and `npx cdk diff` from `CdkBuild/` are the way to see what a change would do before merging it. > [!WARNING] > > Deploying a stage's stacks directly with `cdk deploy` bypasses the image builds and the distribution publish, so the stack can end up referencing image tags that were never pushed. Prefer the pipeline, and use `diff` locally. ## Where Things Are Named Everything is derived from the stage, so the whole estate can be located from it: | Thing | Pattern | | - | - | | Pipeline | `TelemetryPipeline-{stage}` | | Pipeline artifacts bucket | `telemetry-pipeline-bucket-{stage}` | | Data bucket | `telemetry-data-sg-{stage}` | | ECR repository | `telemetry/{image}-{stage}` | | Queues and Lambdas | `{name}-{stage}` | > [!NOTE] > > Production ECR repositories are retained on stack deletion. Dev repositories are destroyed with the stack, as is the pipeline artifacts bucket in both stages. ## Key Files All paths are under `CdkBuild/` in [can-wifi](https://github.com/Smedley-Group-Advanced-Technology/can-wifi). | File | Role | | - | - | | `bin/cdk_build.ts` | The app. Defines the dev and prod stacks | | `lib/cdk_build-stack.ts` | The pipeline itself: source, synth, the image list, the docker build steps and the distribution sync | | `lib/telemetry_ecr_cdk_stack.ts` | The ECR repositories, their scan on push and the keep last 3 rule | | `lib/telemetry_cdk_stack.ts` | The topic, queues, Lambdas, secret and bucket notification | | `lib/build_python_lambdas.ts` | Builds and zips the Python Lambda bundles as a pipeline side input | | `lib/telemetry_utils.ts` | Account and region, the GitHub connection ARN, the SSM lookup and the shared IAM policy documents | ## Related Documentation | Page | Relevance | | - | - | | [Track Operations](/software/developer/estate/systems/telemetry/track-operations) | The other half of a deployment: the setup script pulling at a track | | [DAQ Upload and Conversion](/software/developer/estate/systems/telemetry/daq-pipeline) | What the deployed topic, queue and Lambda actually do with a run | | [Timing and Lap Splitting](/software/developer/estate/systems/telemetry/timing-and-lap-splitting) | The reconversion checker's job, and how a reconversion is triggered | | [The Charging Data Receiver](/software/developer/estate/systems/telemetry/receiver) | What the receiver image does once it is running | This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.