Security Posture
- TLS private key in the repository. The MongoDB server certificate (
mongodb.pem) is committed to the repository, so anyone with repo access has the server private key. The certificate is self-signed, and clients connect with validation relaxed, so the transport encryption is not authenticated. In practice the database is only reachable from inside the VPC through a tight security group, which limits exposure, but the key should not live in source control. - Broad IAM permissions. The MongoDB instance role grants full access to S3 and to Secrets Manager (all actions, all resources), which is wider than needed. The timing provider users are correctly scoped to the ingestion bucket.
- Credentials. The MongoDB root credentials are in Secrets Manager (good). The admin password is also written to a plain file on the instance for the backup job to use.
- Network exposure. MongoDB listens on all interfaces, but the security group only admits the Lambda security group and the instance sits in a private subnet, so it is not internet facing.
Key Risks
- Single point of failure. One EC2 instance and a single node replica set host the core estate database, with no automatic failover.
- No data isolation. TalentID, GKL Platform and Telemetry share the one database with no isolation at the infrastructure level. A problem caused by one system can affect the shared data others rely on.
- Backups with no retention. The nightly backup overwrites the previous one. See Backups and Disaster Recovery.
- No deployment approval. A merge to
maindeploys automatically to this shared production infrastructure. See Architecture and Deployment Pipeline. - Wide blast radius. The VPC and database are shared by almost the whole estate, so a single change here can affect every system at once.
Technical Debt
- Off-standard stack. The repo is Python CDK, while the team standard is TypeScript CDK.
- Pinned base image. The instance AMI is pinned to a specific Amazon Linux 2023 version in
cdk.context.json, so it does not pick up newer images (including security updates) unless changed deliberately. - Fragile resource references. The external dependencies stack reads pre-existing resources by fixed physical names. Some create-if-missing branches can never run, because the lookup always returns a reference, so they are misleading dead code.
- Implicit configuration contract. The SSM parameters downstream systems depend on are an unversioned, implicit interface. See Integration Surface and Configuration Contract.
- Backup coverage lives on the instance, not in the repo. The instance hosts many databases, and the backup script is edited directly on the running instance to cover them. Source control does not reflect the full set of databases backed up, so a rebuild from the repository would under-cover backups until the script is re-edited. See Backups and Disaster Recovery.