> ## Documentation Index
> Fetch the complete documentation index at: https://playbook.smedleygroup.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Shared MongoDB Instance

> The shared MongoDB database for the core estate: a single EC2 instance, how it is provisioned, secured and accessed.

This page describes the shared database that the core estate runs on: a single MongoDB server hosted on an EC2 instance, and how it is built and secured.

## Overview

The `TalentId-MongoInstance` stack provisions one EC2 instance that runs MongoDB, inside the shared VPC. This single instance is the shared database for TalentID:Coaching, TalentID:Global, GKL Platform and Telemetry. It is self-managed: the team is responsible for the operating system, the MongoDB installation, patching, backups and recovery. It is not a managed AWS database service.

## The Instance

| Property | Value |
| - | - |
| Type | EC2 `t2.medium` |
| Operating system | Amazon Linux 2023 (AMI pinned in `cdk.context.json`) |
| Storage | Single 30 GB EBS root volume |
| Placement | A private-with-egress subnet, no public address |
| Name tag | `Mongo Node V72` |

## How It Is Provisioned

The stack uploads seven configuration and script files to S3 as CDK assets, then runs them through EC2 user data on first boot. The main script (`init_instance.sh`) performs the following:

**1. Generate credentials**

Generates two random 128 character passwords, for the `root` and `admin` database users.

**2. Create the secret**

Creates a secret in AWS Secrets Manager (`/talentId/mongodb/secret`) holding the root credentials (host, username `root`, password, port `27017`, ssl true).

**3. Install MongoDB**

Installs MongoDB 7.0 from the official MongoDB yum repository, and installs cron.

**4. Create users**

Starts MongoDB, then creates the `root` and `admin` users.

**5. Configure MongoDB**

Applies `mongod.conf`: TLS required on all connections using the certificate at `/etc/mongodb.pem`, authentication enabled, a key file for internal authentication, and a replica set named `rs0`.

**6. Initiate the replica set**

Generates the internal key file and initiates a single node replica set (`rs0`).

**7. Schedule jobs**

Installs cron jobs for nightly backups and a password resync on reboot. See [Backups and Disaster Recovery](/software/developer/estate/infra/talentidpipeline-infra-cdk/backups).

## Security Model

* **Network:** the MongoDB security group allows inbound only on `27017`, and only from the Lambda security group. MongoDB listens on all interfaces, but is reachable only by approved workloads in the VPC and is not exposed to the internet.
* **Transport:** all client connections require TLS. The certificate is a self-signed certificate shipped with the repository, and clients connect with certificate validation relaxed.
* **Authentication:** enabled. The root credentials live in Secrets Manager and are the source of truth. On every reboot a script re-reads the secret and resets the root password so it stays in sync.
* **Authorisation:** the database users are created with the `root` role.

> [!WARNING]
>
> There is no data isolation between systems at the infrastructure level. TalentID, GKL Platform and Telemetry all use this one instance, which hosts many databases (the dev and prod databases for each system that uses it). There is no separate dev or prod AWS environment; those databases simply live side by side on the same instance. `TalentId_DEV` and `TalentId_PROD` are two examples, named by the backup automation in the repository.

## Why It Is Built This Way

* Self-hosting MongoDB on EC2 was the original approach for TalentID and has been carried forward. It is inexpensive and gives full control, but it puts all the operational burden (patching, scaling, recovery, high availability) on the team.
* It is a single instance and a single node replica set, so it is a single point of failure with no automatic failover. If the instance is lost, the database is restored from the nightly backup onto a new instance.
* The instance name (`Mongo Node V72`) suggests it has been rebuilt many times. Because the data lives on the instance, each rebuild depends on the backup and restore process working.
* A managed database, or at least a multi node replica set with snapshots, would remove most of these risks. Moving systems onto dedicated databases is the direction of the [SG-Infra](/software/developer/estate/infra/sg-infra) pattern.

## Key Files

| File | Role |
| - | - |
| `sg_infra/talent_id_mongo_instance_stack.py` | The EC2 instance, security group, IAM role, key pair and provisioning wiring |
| `sg_infra/mongo_conf/init_instance.sh` | The main provisioning script |
| `sg_infra/mongo_conf/mongod.conf` | MongoDB server config (TLS, auth, replica set) |
| `sg_infra/mongo_conf/mongo_init.js` | Creates the database users |
| `sg_infra/mongo_conf/synch_pwd_at_startup.sh` | Resets the root password from Secrets Manager on reboot |


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.