> ## Documentation Index > Fetch the complete documentation index at: https://playbook.smedleygroup.com/llms.txt > Use this file to discover all available pages before exploring further. # Infrastructure BAU > The recurring operational work that keeps the AWS estate healthy: what is checked, how often, and how to get to it. Active ## Overview > [!NOTE] > > These pages were written in September 2026 by David Shortland. They describe the estate as it was at that point. Infrastructure BAU is the recurring operational work that keeps the AWS estate running between projects. None of it is feature work and none of it is incident response. It is the routine checking that catches problems while they are still cheap, and it exists because almost everything in the estate is self-managed rather than a managed AWS service. The estate has no dedicated operations function. This section is the substitute: a fixed rhythm of weekly, monthly and quarterly checks, currently sized to about a day a week. > [!NOTE] > > This section covers infrastructure only. Application code maintenance is separate, covering dependency and framework updates (for example .NET and Angular package upgrades), CVE triage on those dependencies, and keeping the test suites green. That work is not documented here. ## Key Facts | Setting | Value | | - | - | | Time commitment | Approximately 8 hours per week | | AWS account | `338232497876` | | Region | `eu-west-2` (London) | | Databases in scope | Both MongoDB instances (legacy shared, and SG-Infra) | | Systems in scope | TalentID Coaching, TalentID Global, GKL Platform, Telemetry, Fleet Management Platform | ## How You Connect There are two routes, and which one you use depends on what you are checking. | Checking | Route | | - | - | | Instance level: disk, CPU, memory, instance state | The EC2 console, using **Connect > Session Manager** | | Database level: database sizes, collections, replica set status | Studio 3T on the Windows workstation | For anything on a MongoDB instance itself, open the EC2 console, select the instance, choose **Connect**, then the **Session Manager** tab, and connect from there. That gives you a shell on the instance in the browser, with no keys and nothing installed locally. The Windows workstation (also referred to as the EC2 desktop) is a third EC2 instance in the same VPC as the MongoDB instances, reachable through Session Manager and Remote Desktop. Studio 3T is installed on it, and it is the route to the databases for routine checks such as sizes and growth. > [!WARNING] > > Do not connect to the MongoDB instances directly from your own machine. Database access goes through Studio 3T on the Windows workstation, and instance level checks go through the EC2 console using **Connect > Session Manager**. ### Connecting To The Windows Workstation The workstation is normally kept stopped, so connecting to it is start, use, stop. It sits in a private subnet with no public address, so there is no direct Remote Desktop route to it: Session Manager forwards the Remote Desktop port to a local port on your machine instead. The commands below are PowerShell, and need the AWS CLI and the [Session Manager plugin](https://docs.aws.amazon.com/systems-manager/latest/userguide/session-manager-working-with-install-plugin.html) installed, with a named profile configured for the account and permission to read the credentials secret. | Setting | Value | | - | - | | Instance | `i-0800e15b1bcbfd7f1` | | Region | `eu-west-2` | | Local port | `35901` | | Username | `Administrator` | | Credentials | Secrets Manager, `/Infra/Workstation/rdp` | **1. Start the instance** Start it from the EC2 console, or from PowerShell: ```powershell theme={null} aws ec2 start-instances --instance-ids i-0800e15b1bcbfd7f1 --region eu-west-2 --profile TID ``` Wait for the instance to reach the running state before opening the tunnel. **2. Retrieve the credentials** The username and password are held in Secrets Manager under `/Infra/Workstation/rdp`. Read the secret in the console, or from PowerShell: ```powershell theme={null} aws secretsmanager get-secret-value --secret-id /Infra/Workstation/rdp --region eu-west-2 --profile TID --query SecretString --output text ``` **3. Open the port forwarding session** Leave this running for as long as you need the connection. Closing it closes the tunnel. ```powershell theme={null} aws ssm start-session --target i-0800e15b1bcbfd7f1 --document-name AWS-StartPortForwardingSession --parameters "localPortNumber=35901,portNumber=3389" --region eu-west-2 --profile TID ``` **4. Connect with Remote Desktop** Open Remote Desktop Connection against the forwarded local port, and log in with the credentials from the previous step. ```powershell theme={null} mstsc /v:localhost:35901 ``` **5. Stop the instance when you are finished** Close the port forwarding session, then stop the instance from the EC2 console or from PowerShell: ```powershell theme={null} aws ec2 stop-instances --instance-ids i-0800e15b1bcbfd7f1 --region eu-west-2 --profile TID ``` Confirm it has actually reached the stopped state rather than assuming the command took effect. > [!WARNING] > > The workstation is expensive to run and must be stopped once you are done. Leaving it running is a standing cost for a machine nobody is using, and it will show up in the weekly [cost review](/software/developer/operations/weekly#cost-review) as an unexplained increase. > [!NOTE] > > `--profile TID` refers to a named AWS CLI profile configured locally. Substitute whichever profile name you have configured for the account. ## Why This Is Needed The estate is built almost entirely on self-managed infrastructure, which means the operational burden sits with the team rather than with AWS: * **The databases are self-hosted.** Both MongoDB instances run on EC2 that the team owns end to end: operating system, patching, disk, backups and recovery. See [Shared MongoDB Instance](/software/developer/estate/infra/talentidpipeline-infra-cdk/mongodb). * **Each database is a single point of failure.** Single instances with single node replica sets, no automatic failover. Recovery is a manual restore. * **Backup coverage lives on the instances.** The backup scripts are edited directly on the running instances as databases are added, so source control does not reflect what is actually backed up. See [Backups and Disaster Recovery](/software/developer/estate/infra/talentidpipeline-infra-cdk/backups). * **The blast radius is the whole estate.** One VPC and one shared database serve most systems, so a single problem can affect everything at once. * **Some failures are silent.** A Vivenu webhook that stops delivering, a backup that writes an empty file, or a queue quietly discarding messages will not raise an alarm. They are found by looking. ## The Cadence > **[Weekly Checks](/software/developer/operations/weekly)** > > Database health, backups, Vivenu webhooks, alarms, queues, cost and pipelines. > **[Monthly Deep Dives](/software/developer/operations/monthly)** > > Restore rehearsal, capacity and data growth, and cost optimisation. > **[Quarterly Reviews](/software/developer/operations/quarterly)** > > Access and security review, documentation refresh, and vendor review. ## How To Use This Section Work through [Weekly Checks](/software/developer/operations/weekly) in order once a week. Most weeks nothing will be wrong, and the value is the trend rather than the snapshot: disk that grew faster than usual, a backup that shrank, a cost line that moved. Record what you saw so the following week has something to compare against. > [!WARNING] > > Anything you find that is not covered by these pages should be added to them. The estate has very little operational documentation, and this section is only useful for as long as it reflects what is actually being run. ## Related Documentation | Page | Relevance | | - | - | | [TalentIdPipeline-Infra-CDK Overview](/software/developer/estate/infra/talentidpipeline-infra-cdk) | The shared VPC and legacy MongoDB instance | | [SG-Infra Overview](/software/developer/estate/infra/sg-infra) | The newer pattern and its MongoDB instance | | [Backups and Disaster Recovery](/software/developer/estate/infra/talentidpipeline-infra-cdk/backups) | What the backups do and do not cover | | [Security, Risks and Technical Debt](/software/developer/estate/infra/talentidpipeline-infra-cdk/security) | The standing risk register these checks watch | | [Configuring the Webhooks](/software/third-party/vivenu/hub-deployment/development/configure-webhooks) | How the Vivenu webhooks are set up | This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.