From 72406c54881cab8b5f19f609931eadeae97b7f2e Mon Sep 17 00:00:00 2001 From: Dragos Andriciuc Date: Thu, 8 May 2025 15:50:24 +0300 Subject: [PATCH 01/11] updated replication.md started to populate replication with information and added to ToC --- .../pg_tde/documentation/docs/replication.md | 230 +++++++++++++++++- contrib/pg_tde/documentation/mkdocs.yml | 21 +- 2 files changed, 238 insertions(+), 13 deletions(-) diff --git a/contrib/pg_tde/documentation/docs/replication.md b/contrib/pg_tde/documentation/docs/replication.md index b745107a2f222..fc16e2b3902c9 100644 --- a/contrib/pg_tde/documentation/docs/replication.md +++ b/contrib/pg_tde/documentation/docs/replication.md @@ -1,4 +1,228 @@ -# Replication +# Streaming Replication with tde_heap - -This topic outlines how to set up PostgreSQL streaming replication when `pg_tde` (specifically the `tde_heap` access method) is enabled on the primary server. +# Streaming Replication with pg_tde (heap_tde) -!!! note - Physical streaming replication copies data blocks directly from the primary to the standby. If these blocks are encrypted on the primary, they will arrive encrypted on the standby. Therefore, the standby **must** have access to the **exact same encryption keys** as the primary to decrypt and read this data. +This section outlines how to set up PostgreSQL streaming replication when Percona's `pg_tde` extension (specifically the `heap_tde` access method) is enabled on the primary server. -## 1. Key Concepts +!!! note + Physical streaming replication copies data blocks directly from the primary to the standby. If these blocks are encrypted on the primary, they arrive encrypted on the standby. Therefore, the standby **must** have access to the **exact same encryption keys** as the primary to decrypt and read this data. -* **`tde_heap`:** An access method that encrypts entire tables using a master key. It's simpler to manage in a replication scenario than `tde_heap_basic`. -* **Master Key:** The central key used by `pg_tde` to encrypt/decrypt data encryption keys (DEKs) or, in simpler configurations, directly encrypt data. -* **Key Provider:** The mechanism `pg_tde` uses to fetch the master key (a local file, HashiCorp Vault, and more). -* **Physical Streaming Replication:** Replicates WAL records, which contain block-level changes. Encrypted data blocks are replicated as-is. +## Key Concepts -## 2. How to Set Up Key Provider and Master Key for Replication +* **`pg_tde`:** Percona's extension for Transparent Data Encryption in PostgreSQL. +* **`heap_tde`:** The table access method provided by `pg_tde` that encrypts entire tables using a master key. +* **Master Key:** The central key used by `pg_tde` to encrypt/decrypt data. Identified by a `pg_tde_master_key_name`. +* **Key Provider:** The mechanism `pg_tde` uses to fetch the master key (e.g., local files, HashiCorp Vault). Configured via `pg_tde_key_provider`. -The standby server(s) **must** be configured to use the **same key provider** and access the **same master key** as the primary server. +## How to Set Up Key Provider and Master Key for Replication -### Scenario 1: File-based Key Provider (Example) +The standby server(s) **must** be configured to use the **same key provider settings** and access the **same master key (by name and content)** as the primary server. -If you're using a `keyring_file`: +### Scenario 1: File-based Key Provider 1. **Primary:** - * Configure `pg_tde.key_provider_type = 'file'` (or your chosen file provider type). - * Configure `pg_tde.keyring_file_path = '/path/to/your/master.key'` (or equivalent for your provider). - * Ensure the master key file exists and has appropriate permissions (readable by the PostgreSQL OS user). - * Initialize the master key if it's the first time: `SELECT pg_tde_master_key_init();` + * In `postgresql.conf`: + + ```ini + pg_tde_key_provider = 'file' + pg_tde_file_keys_path = '/var/lib/postgresql/pg_tde_keys' # Example directory + pg_tde_master_key_name = 'my_primary_master_key' + ``` + + * Create the directory: `sudo -u postgres mkdir /var/lib/postgresql/pg_tde_keys` + * `sudo -u postgres chmod 700 /var/lib/postgresql/pg_tde_keys` + * Place your actual master key file (for example, a file containing a 32-byte AES key) inside this directory, named `my_primary_master_key`: + `sudo -u postgres dd if=/dev/urandom of=/var/lib/postgresql/pg_tde_keys/my_primary_master_key bs=32 count=1` + `sudo -u postgres chmod 600 /var/lib/postgresql/pg_tde_keys/my_primary_master_key` + * After restarting PostgreSQL, in `psql`: + + ```sql + CREATE EXTENSION IF NOT EXISTS pg_tde; -- If not already done + -- Add the key definition to pg_tde's catalog (it will read from the file) + SELECT pg_tde_master_key_add('my_primary_master_key', NULL, 'file'); + -- Set this key as the active master key + SELECT pg_tde_set_master_key('my_primary_master_key', 'file'); + ``` + 2. **Standby:** - * Before starting the standby, **securely copy** the *exact same master key file* from the primary to the *exact same path* on the standby (`/path/to/your/master.key`). - * Ensure permissions on the standby's copy are identical (readable by PostgreSQL OS user). - * In the standby's `postgresql.conf`, configure `pg_tde.key_provider_type` and `pg_tde.keyring_file_path` to be **identical** to the primary's configuration. + * Before starting the standby, **securely copy** the entire `pg_tde_file_keys_path` directory and its contents (specifically the `my_primary_master_key` file) from the primary to the exact same path on the standby (`/var/lib/postgresql/pg_tde_keys`). + * Ensure permissions on the standby's directory and key file are identical. + * In the standby's `postgresql.conf`, configure `pg_tde_key_provider`, `pg_tde_file_keys_path`, and `pg_tde_master_key_name` to be **identical** to the primary's configuration. + * The standby, upon startup, uses these settings to load the key. -### Scenario 2: Centralized Key Provider +### Scenario 2: HashiCorp Vault Key Provider** 1. **Primary:** - * Configure `pg_tde.key_provider_type = 'vault'` (or your chosen Vault provider type). - * Configure Vault connection parameters (address, token/auth method, key path in Vault, etc.). - * Ensure the master key exists in Vault at the specified path. - * Initialize if needed: `SELECT pg_tde_master_key_init('your_master_key_name_in_vault');` + * Ensure the master key (e.g., `my_vault_master_key`) exists in Vault at the configured path (for example, `secret/data/postgres/keys`). + * In `postgresql.conf`: + + ```ini + pg_tde_key_provider = 'vault' + pg_tde_vault_url = 'http://your-vault-server:8200' + pg_tde_vault_token = 'your-vault-access-token' # Or use other auth methods + pg_tde_vault_mount_path = 'secret/' # Path where KV engine is mounted + # pg_tde_vault_ca_path = '/path/to/vault_ca.crt' # If using self-signed certs + pg_tde_master_key_name = 'my_vault_master_key' # Key name within Vault path + ``` + + * After restarting PostgreSQL, in `psql`: + + ```sql + CREATE EXTENSION IF NOT EXISTS pg_tde; + -- Add the key definition (pg_tde will fetch it from Vault) + SELECT pg_tde_master_key_add('my_vault_master_key', NULL, 'vault'); + -- Set this key as the active master key + SELECT pg_tde_set_master_key('my_vault_master_key', 'vault'); + ``` + 2. **Standby:** - * In the standby's `postgresql.conf`, configure `pg_tde.key_provider_type` and all Vault connection parameters to be **identical** to the primary's configuration. + * In the standby's `postgresql.conf`, configure `pg_tde_key_provider` and all `pg_tde_vault_*` parameters, and `pg_tde_master_key_name` to be **identical** to the primary's configuration. * Ensure the standby server has network access to the Vault server and the necessary credentials/permissions to read the *same master key* from Vault. - * When the standby starts, it will use these settings to fetch the master key from Vault. + * The standby, upon startup, uses these settings to load the key. **Is it possible to use the same key provider and master key?** -**Yes, it is mandatory.** For physical replication to function with `pg_tde`, the standby *must* use the same master key as the primary to decrypt the replicated data blocks. This implies using a key provider setup that allows the standby to access this same key. +**Yes, it is mandatory.** For physical replication to function with TDE, the standby *must* use the same master key (by name and content, accessed via the same provider configuration) as the primary to decrypt the replicated data blocks. -## 3. Encrypted Table Replication +## Encrypted Table Replication **If a table on the primary site is encrypted, will all the standby sites also have the encrypted table replicated, given that the same key is implemented on the standby sites as well?** Yes. + Physical replication works at the block level. -1. When you encrypt a table on the primary using `pg_tde`, the data blocks on disk are encrypted. -2. When data in this table is modified, the changes (which are also encrypted) are written to WAL. -3. Streaming replication sends these WAL records (containing encrypted block changes) to the standby. -4. The standby applies these WAL records, writing the encrypted blocks to its own disk. -5. If the standby has the same master key available via its configured key provider, it can then decrypt these blocks when you query the table on the standby. +1. When you encrypt a table on the primary using `CREATE TABLE ... USING heap_tde;` or by setting `default_table_access_method = 'heap_tde'`, the data blocks on disk are encrypted. +2. Changes to this table (also encrypted) are written to WAL. +3. Streaming replication sends these WAL records to the standby. +4. The standby applies these WAL records, writing the encrypted blocks to its disk. +5. If the standby has the same master key available (via identical `pg_tde` configuration and key access), it can decrypt these blocks when you query the table on the standby. -If the standby *does not* have the correct key, it still replicates the encrypted data, but it is unable to read it, and queries against encrypted tables fail. +If the standby lacks the correct key, replicated data remains encrypted and unreadable, and queries will fail. -## 4. Setup Steps +## Setup Steps -This assumes you have the `pg_tde` extension installed on both primary and standby. +This assumes the `pg_tde` extension is available for installation on both primary and standby. ### A. Primary Server Setup -1. **Configure postgresql.conf for pg_tde:** +1. **Install the `pg_tde` extension files.** +2. **Configure `postgresql.conf` for `pg_tde`:** ```ini - shared_preload_libraries = 'pg_tde' - pg_tde.key_provider_type = 'file' # Or 'vault', etc. - pg_tde.keyring_file_path = '/var/lib/postgresql/data/pg_tde_master.key' # Example for file - # If using Vault, add Vault-specific GUCs - # pg_tde.vault_addr = 'http://your-vault-server:8200' - # pg_tde.vault_token = 'your-vault-token-or-auth-path' - # pg_tde.vault_key_path = 'secret/data/postgres/master_key' - # pg_tde.master_key_name = 'my_pg_master_key' # Name for the key in Vault + shared_preload_libraries = 'pg_tde' // Add pg_tde, comma-separated if others exist + + # Example for File Provider: + pg_tde_key_provider = 'file' + pg_tde_file_keys_path = '/var/lib/postgresql/pg_tde_keys' + pg_tde_master_key_name = 'my_master_key' + + # Example for Vault Provider: + # pg_tde_key_provider = 'vault' + # pg_tde_vault_url = 'http://your-vault-server:8200' + # pg_tde_vault_token = 's.xxxxxxxxxxxxxx' + # pg_tde_vault_mount_path = 'secret/' + # pg_tde_master_key_name = 'my_master_key_in_vault' ``` -2. **Configure postgresql.conf for Replication:** +3. **Configure `postgresql.conf` for Replication:** ```ini wal_level = replica max_wal_senders = 5 - archive_mode = on # Optional, but good practice - archive_command = 'cp %p /path/to/archive/%f' # Example archive command + archive_mode = on # Recommended + archive_command = 'cp %p /path/to/archive/%f' # Example ``` -3. **Create Master Key (if it does not exist):** - * **For File Provider:** Create an empty file or one with your key if you have it. - `sudo -u postgres touch /var/lib/postgresql/data/pg_tde_master.key` - `sudo -u postgres chmod 600 /var/lib/postgresql/data/pg_tde_master.key` - * **For Vault Provider:** Ensure the key exists in Vault at the configured path and name. -4. **Restart PostgreSQL on Primary.** -5. **Initialize Master Key in psql (if first time):** +4. **Prepare Master Key in Provider:** - ```sql - -- For file provider (key name is usually 'default' or derived from filename) - SELECT pg_tde_master_key_init(); - -- For Vault provider (specify key name) - -- SELECT pg_tde_master_key_init('my_pg_master_key'); - ``` + * **File:** Create the directory specified in `pg_tde_file_keys_path`. Create the key file inside this directory, named as per `pg_tde_master_key_name`. + + ```bash + sudo -u postgres mkdir -p /var/lib/postgresql/pg_tde_keys + sudo -u postgres chmod 700 /var/lib/postgresql/pg_tde_keys + sudo -u postgres dd if=/dev/urandom of=/var/lib/postgresql/pg_tde_keys/my_master_key bs=32 count=1 + sudo -u postgres chmod 600 /var/lib/postgresql/pg_tde_keys/my_master_key + ``` -6. **Enable the pg_tde extension and set tde_heap as default:** + * **Vault:** Ensure the key exists in Vault at the correct path and is accessible. +5. **Restart PostgreSQL on Primary.** +6. **Initialize `pg_tde` and Set Master Key (in `psql`):** ```sql CREATE EXTENSION pg_tde; - ALTER SYSTEM SET default_table_access_method = 'tde_heap'; + -- Add the key definition to pg_tde's catalog + -- For File (key_name matches pg_tde_master_key_name and filename): + SELECT pg_tde_master_key_add('my_master_key', NULL, 'file'); + -- For Vault (key_name matches pg_tde_master_key_name and key name in Vault): + -- SELECT pg_tde_master_key_add('my_master_key_in_vault', NULL, 'vault'); + + -- Set this key as the active master key + -- For File: + SELECT pg_tde_set_master_key('my_master_key', 'file'); + -- For Vault: + -- SELECT pg_tde_set_master_key('my_master_key_in_vault', 'vault'); + + -- Set heap_tde as default for new tables (optional) + ALTER SYSTEM SET default_table_access_method = 'heap_tde'; SELECT pg_reload_conf(); ``` -7. **Create a Replication User:** - - ```sql - CREATE ROLE replicator REPLICATION LOGIN PASSWORD 'strong_password'; - ``` - -8. **Configure pg_hba.conf to allow replication connection:** - - ``` - host replication replicator your_standby_ip/32 scram-sha-256 - ``` - - Reload `pg_hba.conf`: `SELECT pg_reload_conf();` - -9. **Create and Encrypt a Test Table:** +7. **Create Replication User & Configure `pg_hba.conf` (standard PostgreSQL steps).** +8. **Create and Encrypt a Test Table:** ```sql - CREATE TABLE encrypted_stuff (id int primary key, secret_data text); - -- It will be automatically encrypted if default_table_access_method = 'tde_heap' - -- Or explicitly: CREATE TABLE encrypted_stuff (...) USING tde_heap; + -- If default_table_access_method = 'heap_tde' + CREATE TABLE encrypted_data (id INT PRIMARY KEY, data TEXT); + -- Or explicitly: + -- CREATE TABLE encrypted_data (id INT PRIMARY KEY, data TEXT) USING heap_tde; - INSERT INTO encrypted_stuff VALUES (1, 'My secret data on primary'); + INSERT INTO encrypted_data VALUES (1, 'Secret on primary'); ``` -### B. Standby Server Setup - -1. **Install PostgreSQL and the pg_tde extension.** -2. **Configure postgresql.conf for pg_tde - IDENTICAL to Primary:** - Copy the `pg_tde.*` GUCs from the primary's `postgresql.conf` to the standby's `postgresql.conf`. - - ```ini - shared_preload_libraries = 'pg_tde' - pg_tde.key_provider_type = 'file' - pg_tde.keyring_file_path = '/var/lib/postgresql/data/pg_tde_master.key' - # Or IDENTICAL Vault settings - ``` +### B. Standby Server Setup** +1. **Install `pg_tde` extension files.** +2. **Configure `postgresql.conf` for `pg_tde` - IDENTICAL to Primary:** + Copy the `pg_tde_*` GUCs from the primary's `postgresql.conf` to the standby's `postgresql.conf`. Ensure they are exactly the same. 3. **Ensure Master Key Accessibility:** - * **For File Provider:** Securely copy the *exact same master key file* from primary to the standby at the *exact same path* (`/var/lib/postgresql/data/pg_tde_master.key`). Ensure permissions are `600` and owned by the `postgres` user. - * **For Vault Provider:** Ensure the standby has network connectivity and credentials to access the *same key name* in Vault. + * **File Provider:** Securely copy the *entire key directory* (e.g., `/var/lib/postgresql/pg_tde_keys`) and its contents from primary to the standby at the *exact same path*. Ensure permissions are identical. + * **Vault Provider:** Ensure standby has network and credential access to the *same key name* in Vault. 4. **Stop Standby PostgreSQL service (if running).** 5. **Remove any existing data from the standby's data directory.** 6. **Take a Base Backup from Primary:** ```bash # Run as postgres OS user on the standby server - pg_basebackup -h your_primary_ip -U replicator -p 5432 -D /var/lib/postgresql/your_data_dir -Fp -Xs -P -R - # -R creates standby.signal and primary_conninfo in postgresql.auto.conf - # (For older PG versions, you might need to manually create recovery.conf) + pg_basebackup -h your_primary_ip -U replicator_user -p 5432 -D /path/to/standby_data_dir -Fp -Xs -P -R ``` - Enter the password for `replicator` when prompted. - 7. **Start PostgreSQL on Standby.** - The standby will connect to the primary, start streaming WAL, and apply changes. Because it has the same `pg_tde` configuration and access to the same master key, it will be able to decrypt data. + The standby will connect, stream WAL, and apply changes. With identical `pg_tde` config and key access, it will decrypt data. -## 5. Test the Implementation +## Test the Implementation -1. **Follow all steps in Section 4 (A and B) meticulously.** +1. **Follow all steps in Section 4 (A and B) precisely.** 2. **On the Primary:** - * Connect via `psql`. - * Verify the table exists and data can be queried: - - ```sql - SELECT * FROM encrypted_stuff; - -- Expected: (1, 'My secret data on primary') - ``` - - * Insert more data: + * Insert more data into `encrypted_data`. ```sql - INSERT INTO encrypted_stuff VALUES (2, 'Another secret from primary'); + INSERT INTO encrypted_data VALUES (2, 'More secrets from primary'); ``` 3. **On the Standby:** - * Wait a few moments for replication to catch up. - * Connect via `psql`. - * Check replication status (optional, but good). + * Wait for replication. Connect via `psql`. + * **Crucial Test:** Query the encrypted table: ```sql - SELECT pg_is_wal_replay_paused(); -- should be false - SELECT pg_last_wal_receive_lsn(), pg_last_wal_replay_lsn(); -- should be advancing + SELECT * FROM encrypted_data; ``` - * **Crucial Test:** Query the encrypted table. + **Expected Result:** You should see all data, including new rows, decrypted successfully. ```sql - SELECT * FROM encrypted_stuff; - ``` - - **Expected Result:** You should see all data, including the new row. - - ``` - id | secret_data - ----+----------------------------- - 1 | My secret data on primary - 2 | Another secret from primary + id | data + ----+------------------------- + 1 | Secret on primary + 2 | More secrets from primary (2 rows) ``` - If you can query the data successfully, it means the standby was able to decrypt the replicated encrypted blocks using the master key. - * **Negative Test (Optional):** Temporarily make the master key unavailable on the standby (e.g., rename the key file if using file provider, or revoke Vault access) and restart the standby. Queries to `encrypted_stuff` should now fail with a TDE-related error. Restore the key and restart to confirm functionality returns. - -## 6. Limitations and Important Considerations - -* **Master Key is Critical:** Loss of the master key means loss of all TDE-encrypted data. Back it up securely and separately. -* **Identical Configuration:** `pg_tde` configuration GUCs (especially those related to key provider and key name/path) **must** be identical on primary and all standbys. -* **Key Provider Accessibility:** The chosen key provider must be accessible from all standby servers with the necessary permissions to fetch the *same* master key. -* **Key Rotation:** - * When a master key is rotated on the primary, `pg_tde` will manage this. New data is encrypted with the new key (or references to it). - * For standbys to continue functioning, they must also gain access to this new master key via the key provider. This is usually seamless if using a centralized provider like Vault where the key name remains the same but its version/content changes. For file-based keys, you need to securely distribute the new key file to standbys. - * Existing data blocks are not typically re-encrypted immediately upon master key rotation. They are re-encrypted when the data pages are next written (e.g., due to an `UPDATE`, `VACUUM FULL`, or `pg_tde_rotate_key_data()` type functions if available for `tde_heap`). -* **Complexity with `tde_heap_basic`:** If you were using `tde_heap_basic` which allows per-table/per-column keys, key management across replicas could be more complex unless all keys are centrally managed and referenced by name from a key provider. `tde_heap` simplifies this by relying on a single master key for table encryption. -* **Promotion:** If a standby is promoted to a new primary, it already has the necessary `pg_tde` configuration and key access, so it will function correctly as a TDE-enabled primary. New standbys built from it will need the same key setup. + * **Negative Test (Optional):** Temporarily make the master key file inaccessible on the standby (e.g., rename `pg_tde_file_keys_path` directory or the key file within it) or revoke Vault token. Restart standby. Queries to `encrypted_data` should now fail with a TDE key access error. Restore access and restart to confirm functionality. + +## Limitations and Important Considerations + +* **Master Key Security:** The master key is paramount. Securely back it up. For file provider, protect the key file and its directory stringently. +* **Identical Configuration:** All `pg_tde_*` GUCs relevant to key provider and key name **must** be identical on primary and all standbys. +* **Key Rotation (`pg_tde_rotate_master_key`):** + 1. Create/place the new key in your chosen key provider (new file in `pg_tde_file_keys_path` or new key in Vault). + 2. On the primary, add the new key to `pg_tde`'s catalog: + `SELECT pg_tde_master_key_add('new_master_key_name', NULL, 'your_provider');` + 3. Perform rotation: + `SELECT pg_tde_rotate_master_key('old_master_key_name', 'new_master_key_name', 'your_provider');` + This re-encrypts the internal DEKs with the new master key. Data files are re-encrypted lazily or via `pg_tde_encrypt_table()`. + 4. Update `pg_tde_master_key_name` in `postgresql.conf` on the primary to `'new_master_key_name'`. + 5. **Crucially for replication:** + * Ensure the `new_master_key_name` (and its actual key material) is distributed/accessible to all standbys. + * Update `pg_tde_master_key_name` in `postgresql.conf` on all standbys to match the primary. + * A rolling restart of standbys might be necessary after their config and key material are updated. +* **Promotion:** A standby promoted to primary will already have the necessary `pg_tde` setup and key access to function as a TDE-enabled primary. From 6d1f228101b86f89b4bda5e37c37cdbd5136d702 Mon Sep 17 00:00:00 2001 From: Dragos Andriciuc Date: Thu, 8 May 2025 17:28:19 +0300 Subject: [PATCH 03/11] small fixes to text Updated text a bit, need to further lint it more PG-802 linked --- contrib/pg_tde/documentation/docs/replication.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/contrib/pg_tde/documentation/docs/replication.md b/contrib/pg_tde/documentation/docs/replication.md index cc79411c180dd..d14b92a343df7 100644 --- a/contrib/pg_tde/documentation/docs/replication.md +++ b/contrib/pg_tde/documentation/docs/replication.md @@ -13,16 +13,16 @@ This section outlines how to set up PostgreSQL streaming replication when Percona's `pg_tde` extension (specifically the `heap_tde` access method) is enabled on the primary server. !!! note - Physical streaming replication copies data blocks directly from the primary to the standby. If these blocks are encrypted on the primary, they arrive encrypted on the standby. Therefore, the standby **must** have access to the **exact same encryption keys** as the primary to decrypt and read this data. + Physical streaming replication copies data blocks directly from the primary to the standby. If these blocks are encrypted on the primary, they arrive **encrypted** on the standby. Therefore, the standby **must** have access to the **exact same encryption keys** as the primary to decrypt and read this data. -## Key Concepts +## Concepts * **`pg_tde`:** Percona's extension for Transparent Data Encryption in PostgreSQL. * **`heap_tde`:** The table access method provided by `pg_tde` that encrypts entire tables using a master key. * **Master Key:** The central key used by `pg_tde` to encrypt/decrypt data. Identified by a `pg_tde_master_key_name`. * **Key Provider:** The mechanism `pg_tde` uses to fetch the master key (e.g., local files, HashiCorp Vault). Configured via `pg_tde_key_provider`. -## How to Set Up Key Provider and Master Key for Replication +## Set Up Key Provider and Master Key for Replication The standby server(s) **must** be configured to use the **same key provider settings** and access the **same master key (by name and content)** as the primary server. From e6722416306d0917263c5025e066d197bf2e545b Mon Sep 17 00:00:00 2001 From: Dragos Andriciuc Date: Wed, 21 May 2025 09:42:34 +0300 Subject: [PATCH 04/11] Updated replication.md Replaced master with principal --- .../pg_tde/documentation/docs/replication.md | 102 +++++++++--------- 1 file changed, 51 insertions(+), 51 deletions(-) diff --git a/contrib/pg_tde/documentation/docs/replication.md b/contrib/pg_tde/documentation/docs/replication.md index d14b92a343df7..cb5c88e0e78a3 100644 --- a/contrib/pg_tde/documentation/docs/replication.md +++ b/contrib/pg_tde/documentation/docs/replication.md @@ -2,10 +2,10 @@ * **Table Access Method:** The encrypted table access method is `heap_tde`. * **Key Provider GUC:** `pg_tde_key_provider` (e.g., `'file'`, `'vault'`). * **File Provider Path GUC:** `pg_tde_file_keys_path` (points to a *directory* where key files are stored, not a single key file). -* **Master Key Name GUC:** `pg_tde_master_key_name` (this is the logical name of the key, which then maps to a file in `pg_tde_file_keys_path` or an entry in Vault). +* **Principal Key Name GUC:** `pg_tde_principal_key_name` (this is the logical name of the key, which then maps to a file in `pg_tde_file_keys_path` or an entry in Vault). * **Key Management Functions:** - * `pg_tde_master_key_add(key_name TEXT, key_value TEXT, key_provider TEXT)`: Adds a key definition. `key_value` can be `NULL` if fetching from an external provider. - * `pg_tde_set_master_key(key_name TEXT, provider_name TEXT)`: Sets the active master key for the cluster. + * `pg_tde_principal_key_add(key_name TEXT, key_value TEXT, key_provider TEXT)`: Adds a key definition. `key_value` can be `NULL` if fetching from an external provider. + * `pg_tde_set_principal_key(key_name TEXT, provider_name TEXT)`: Sets the active principal key for the cluster. * **Vault GUCs:** `pg_tde_vault_url`, `pg_tde_vault_token`, `pg_tde_vault_mount_path`, `pg_tde_vault_ca_path`. --> # Streaming Replication with pg_tde (heap_tde) @@ -18,13 +18,13 @@ This section outlines how to set up PostgreSQL streaming replication when Percon ## Concepts * **`pg_tde`:** Percona's extension for Transparent Data Encryption in PostgreSQL. -* **`heap_tde`:** The table access method provided by `pg_tde` that encrypts entire tables using a master key. -* **Master Key:** The central key used by `pg_tde` to encrypt/decrypt data. Identified by a `pg_tde_master_key_name`. -* **Key Provider:** The mechanism `pg_tde` uses to fetch the master key (e.g., local files, HashiCorp Vault). Configured via `pg_tde_key_provider`. +* **`heap_tde`:** The table access method provided by `pg_tde` that encrypts entire tables using a principal key. +* **principal Key:** The central key used by `pg_tde` to encrypt/decrypt data. Identified by a `pg_tde_principal_key_name`. +* **Key Provider:** The mechanism `pg_tde` uses to fetch the principal key (e.g., local files, HashiCorp Vault). Configured via `pg_tde_key_provider`. -## Set Up Key Provider and Master Key for Replication +## Set Up Key Provider and principal Key for Replication -The standby server(s) **must** be configured to use the **same key provider settings** and access the **same master key (by name and content)** as the primary server. +The standby server(s) **must** be configured to use the **same key provider settings** and access the **same principal key (by name and content)** as the primary server. ### Scenario 1: File-based Key Provider @@ -34,34 +34,34 @@ The standby server(s) **must** be configured to use the **same key provider sett ```ini pg_tde_key_provider = 'file' pg_tde_file_keys_path = '/var/lib/postgresql/pg_tde_keys' # Example directory - pg_tde_master_key_name = 'my_primary_master_key' + pg_tde_principal_key_name = 'my_primary_principal_key' ``` * Create the directory: `sudo -u postgres mkdir /var/lib/postgresql/pg_tde_keys` * `sudo -u postgres chmod 700 /var/lib/postgresql/pg_tde_keys` - * Place your actual master key file (for example, a file containing a 32-byte AES key) inside this directory, named `my_primary_master_key`: - `sudo -u postgres dd if=/dev/urandom of=/var/lib/postgresql/pg_tde_keys/my_primary_master_key bs=32 count=1` - `sudo -u postgres chmod 600 /var/lib/postgresql/pg_tde_keys/my_primary_master_key` + * Place your actual principal key file (for example, a file containing a 32-byte AES key) inside this directory, named `my_primary_principal_key`: + `sudo -u postgres dd if=/dev/urandom of=/var/lib/postgresql/pg_tde_keys/my_primary_principal_key bs=32 count=1` + `sudo -u postgres chmod 600 /var/lib/postgresql/pg_tde_keys/my_primary_principal_key` * After restarting PostgreSQL, in `psql`: ```sql CREATE EXTENSION IF NOT EXISTS pg_tde; -- If not already done -- Add the key definition to pg_tde's catalog (it will read from the file) - SELECT pg_tde_master_key_add('my_primary_master_key', NULL, 'file'); - -- Set this key as the active master key - SELECT pg_tde_set_master_key('my_primary_master_key', 'file'); + SELECT pg_tde_principal_key_add('my_primary_principal_key', NULL, 'file'); + -- Set this key as the active principal key + SELECT pg_tde_set_principal_key('my_primary_principal_key', 'file'); ``` 2. **Standby:** - * Before starting the standby, **securely copy** the entire `pg_tde_file_keys_path` directory and its contents (specifically the `my_primary_master_key` file) from the primary to the exact same path on the standby (`/var/lib/postgresql/pg_tde_keys`). + * Before starting the standby, **securely copy** the entire `pg_tde_file_keys_path` directory and its contents (specifically the `my_primary_principal_key` file) from the primary to the exact same path on the standby (`/var/lib/postgresql/pg_tde_keys`). * Ensure permissions on the standby's directory and key file are identical. - * In the standby's `postgresql.conf`, configure `pg_tde_key_provider`, `pg_tde_file_keys_path`, and `pg_tde_master_key_name` to be **identical** to the primary's configuration. + * In the standby's `postgresql.conf`, configure `pg_tde_key_provider`, `pg_tde_file_keys_path`, and `pg_tde_principal_key_name` to be **identical** to the primary's configuration. * The standby, upon startup, uses these settings to load the key. ### Scenario 2: HashiCorp Vault Key Provider** 1. **Primary:** - * Ensure the master key (e.g., `my_vault_master_key`) exists in Vault at the configured path (for example, `secret/data/postgres/keys`). + * Ensure the principal key (e.g., `my_vault_principal_key`) exists in Vault at the configured path (for example, `secret/data/postgres/keys`). * In `postgresql.conf`: ```ini @@ -70,7 +70,7 @@ The standby server(s) **must** be configured to use the **same key provider sett pg_tde_vault_token = 'your-vault-access-token' # Or use other auth methods pg_tde_vault_mount_path = 'secret/' # Path where KV engine is mounted # pg_tde_vault_ca_path = '/path/to/vault_ca.crt' # If using self-signed certs - pg_tde_master_key_name = 'my_vault_master_key' # Key name within Vault path + pg_tde_principal_key_name = 'my_vault_principal_key' # Key name within Vault path ``` * After restarting PostgreSQL, in `psql`: @@ -78,19 +78,19 @@ The standby server(s) **must** be configured to use the **same key provider sett ```sql CREATE EXTENSION IF NOT EXISTS pg_tde; -- Add the key definition (pg_tde will fetch it from Vault) - SELECT pg_tde_master_key_add('my_vault_master_key', NULL, 'vault'); - -- Set this key as the active master key - SELECT pg_tde_set_master_key('my_vault_master_key', 'vault'); + SELECT pg_tde_principal_key_add('my_vault_principal_key', NULL, 'vault'); + -- Set this key as the active principal key + SELECT pg_tde_set_principal_key('my_vault_principal_key', 'vault'); ``` 2. **Standby:** - * In the standby's `postgresql.conf`, configure `pg_tde_key_provider` and all `pg_tde_vault_*` parameters, and `pg_tde_master_key_name` to be **identical** to the primary's configuration. - * Ensure the standby server has network access to the Vault server and the necessary credentials/permissions to read the *same master key* from Vault. + * In the standby's `postgresql.conf`, configure `pg_tde_key_provider` and all `pg_tde_vault_*` parameters, and `pg_tde_principal_key_name` to be **identical** to the primary's configuration. + * Ensure the standby server has network access to the Vault server and the necessary credentials/permissions to read the *same principal key* from Vault. * The standby, upon startup, uses these settings to load the key. -**Is it possible to use the same key provider and master key?** +**Is it possible to use the same key provider and principal key?** -**Yes, it is mandatory.** For physical replication to function with TDE, the standby *must* use the same master key (by name and content, accessed via the same provider configuration) as the primary to decrypt the replicated data blocks. +**Yes, it is mandatory.** For physical replication to function with TDE, the standby *must* use the same principal key (by name and content, accessed via the same provider configuration) as the primary to decrypt the replicated data blocks. ## Encrypted Table Replication @@ -104,7 +104,7 @@ Physical replication works at the block level. 2. Changes to this table (also encrypted) are written to WAL. 3. Streaming replication sends these WAL records to the standby. 4. The standby applies these WAL records, writing the encrypted blocks to its disk. -5. If the standby has the same master key available (via identical `pg_tde` configuration and key access), it can decrypt these blocks when you query the table on the standby. +5. If the standby has the same principal key available (via identical `pg_tde` configuration and key access), it can decrypt these blocks when you query the table on the standby. If the standby lacks the correct key, replicated data remains encrypted and unreadable, and queries will fail. @@ -123,14 +123,14 @@ This assumes the `pg_tde` extension is available for installation on both primar # Example for File Provider: pg_tde_key_provider = 'file' pg_tde_file_keys_path = '/var/lib/postgresql/pg_tde_keys' - pg_tde_master_key_name = 'my_master_key' + pg_tde_principal_key_name = 'my_principal_key' # Example for Vault Provider: # pg_tde_key_provider = 'vault' # pg_tde_vault_url = 'http://your-vault-server:8200' # pg_tde_vault_token = 's.xxxxxxxxxxxxxx' # pg_tde_vault_mount_path = 'secret/' - # pg_tde_master_key_name = 'my_master_key_in_vault' + # pg_tde_principal_key_name = 'my_principal_key_in_vault' ``` 3. **Configure `postgresql.conf` for Replication:** @@ -142,34 +142,34 @@ This assumes the `pg_tde` extension is available for installation on both primar archive_command = 'cp %p /path/to/archive/%f' # Example ``` -4. **Prepare Master Key in Provider:** +4. **Prepare principal Key in Provider:** - * **File:** Create the directory specified in `pg_tde_file_keys_path`. Create the key file inside this directory, named as per `pg_tde_master_key_name`. + * **File:** Create the directory specified in `pg_tde_file_keys_path`. Create the key file inside this directory, named as per `pg_tde_principal_key_name`. ```bash sudo -u postgres mkdir -p /var/lib/postgresql/pg_tde_keys sudo -u postgres chmod 700 /var/lib/postgresql/pg_tde_keys - sudo -u postgres dd if=/dev/urandom of=/var/lib/postgresql/pg_tde_keys/my_master_key bs=32 count=1 - sudo -u postgres chmod 600 /var/lib/postgresql/pg_tde_keys/my_master_key + sudo -u postgres dd if=/dev/urandom of=/var/lib/postgresql/pg_tde_keys/my_principal_key bs=32 count=1 + sudo -u postgres chmod 600 /var/lib/postgresql/pg_tde_keys/my_principal_key ``` * **Vault:** Ensure the key exists in Vault at the correct path and is accessible. 5. **Restart PostgreSQL on Primary.** -6. **Initialize `pg_tde` and Set Master Key (in `psql`):** +6. **Initialize `pg_tde` and Set principal Key (in `psql`):** ```sql CREATE EXTENSION pg_tde; -- Add the key definition to pg_tde's catalog - -- For File (key_name matches pg_tde_master_key_name and filename): - SELECT pg_tde_master_key_add('my_master_key', NULL, 'file'); - -- For Vault (key_name matches pg_tde_master_key_name and key name in Vault): - -- SELECT pg_tde_master_key_add('my_master_key_in_vault', NULL, 'vault'); + -- For File (key_name matches pg_tde_principal_key_name and filename): + SELECT pg_tde_principal_key_add('my_principal_key', NULL, 'file'); + -- For Vault (key_name matches pg_tde_principal_key_name and key name in Vault): + -- SELECT pg_tde_principal_key_add('my_principal_key_in_vault', NULL, 'vault'); - -- Set this key as the active master key + -- Set this key as the active principal key -- For File: - SELECT pg_tde_set_master_key('my_master_key', 'file'); + SELECT pg_tde_set_principal_key('my_principal_key', 'file'); -- For Vault: - -- SELECT pg_tde_set_master_key('my_master_key_in_vault', 'vault'); + -- SELECT pg_tde_set_principal_key('my_principal_key_in_vault', 'vault'); -- Set heap_tde as default for new tables (optional) ALTER SYSTEM SET default_table_access_method = 'heap_tde'; @@ -193,7 +193,7 @@ This assumes the `pg_tde` extension is available for installation on both primar 1. **Install `pg_tde` extension files.** 2. **Configure `postgresql.conf` for `pg_tde` - IDENTICAL to Primary:** Copy the `pg_tde_*` GUCs from the primary's `postgresql.conf` to the standby's `postgresql.conf`. Ensure they are exactly the same. -3. **Ensure Master Key Accessibility:** +3. **Ensure principal Key Accessibility:** * **File Provider:** Securely copy the *entire key directory* (e.g., `/var/lib/postgresql/pg_tde_keys`) and its contents from primary to the standby at the *exact same path*. Ensure permissions are identical. * **Vault Provider:** Ensure standby has network and credential access to the *same key name* in Vault. 4. **Stop Standby PostgreSQL service (if running).** @@ -236,22 +236,22 @@ This assumes the `pg_tde` extension is available for installation on both primar (2 rows) ``` - * **Negative Test (Optional):** Temporarily make the master key file inaccessible on the standby (e.g., rename `pg_tde_file_keys_path` directory or the key file within it) or revoke Vault token. Restart standby. Queries to `encrypted_data` should now fail with a TDE key access error. Restore access and restart to confirm functionality. + * **Negative Test (Optional):** Temporarily make the principal key file inaccessible on the standby (e.g., rename `pg_tde_file_keys_path` directory or the key file within it) or revoke Vault token. Restart standby. Queries to `encrypted_data` should now fail with a TDE key access error. Restore access and restart to confirm functionality. ## Limitations and Important Considerations -* **Master Key Security:** The master key is paramount. Securely back it up. For file provider, protect the key file and its directory stringently. +* **principal Key Security:** The principal key is paramount. Securely back it up. For file provider, protect the key file and its directory stringently. * **Identical Configuration:** All `pg_tde_*` GUCs relevant to key provider and key name **must** be identical on primary and all standbys. -* **Key Rotation (`pg_tde_rotate_master_key`):** +* **Key Rotation (`pg_tde_rotate_principal_key`):** 1. Create/place the new key in your chosen key provider (new file in `pg_tde_file_keys_path` or new key in Vault). 2. On the primary, add the new key to `pg_tde`'s catalog: - `SELECT pg_tde_master_key_add('new_master_key_name', NULL, 'your_provider');` + `SELECT pg_tde_principal_key_add('new_principal_key_name', NULL, 'your_provider');` 3. Perform rotation: - `SELECT pg_tde_rotate_master_key('old_master_key_name', 'new_master_key_name', 'your_provider');` - This re-encrypts the internal DEKs with the new master key. Data files are re-encrypted lazily or via `pg_tde_encrypt_table()`. - 4. Update `pg_tde_master_key_name` in `postgresql.conf` on the primary to `'new_master_key_name'`. + `SELECT pg_tde_rotate_principal_key('old_principal_key_name', 'new_principal_key_name', 'your_provider');` + This re-encrypts the internal DEKs with the new principal key. Data files are re-encrypted lazily or via `pg_tde_encrypt_table()`. + 4. Update `pg_tde_principal_key_name` in `postgresql.conf` on the primary to `'new_principal_key_name'`. 5. **Crucially for replication:** - * Ensure the `new_master_key_name` (and its actual key material) is distributed/accessible to all standbys. - * Update `pg_tde_master_key_name` in `postgresql.conf` on all standbys to match the primary. + * Ensure the `new_principal_key_name` (and its actual key material) is distributed/accessible to all standbys. + * Update `pg_tde_principal_key_name` in `postgresql.conf` on all standbys to match the primary. * A rolling restart of standbys might be necessary after their config and key material are updated. * **Promotion:** A standby promoted to primary will already have the necessary `pg_tde` setup and key access to function as a TDE-enabled primary. From 9e04c3d0ca47546f8f08d0f639ede7c7da27840b Mon Sep 17 00:00:00 2001 From: Dragos Andriciuc Date: Fri, 23 May 2025 15:47:05 +0300 Subject: [PATCH 05/11] updated replication.md with proper information Initial commit was messy, reworded for basic users and cleared up any confusing parts like heap_tde which was incorrect --- .../pg_tde/documentation/docs/replication.md | 321 ++++++------------ 1 file changed, 108 insertions(+), 213 deletions(-) diff --git a/contrib/pg_tde/documentation/docs/replication.md b/contrib/pg_tde/documentation/docs/replication.md index cb5c88e0e78a3..0066bdb6b447a 100644 --- a/contrib/pg_tde/documentation/docs/replication.md +++ b/contrib/pg_tde/documentation/docs/replication.md @@ -1,257 +1,152 @@ - +# Streaming Replication with tde_heap -# Streaming Replication with pg_tde (heap_tde) +This section outlines how to set up PostgreSQL streaming replication when the `pg_tde` extension (specifically the `tde_heap` access method) is enabled on the primary server. -This section outlines how to set up PostgreSQL streaming replication when Percona's `pg_tde` extension (specifically the `heap_tde` access method) is enabled on the primary server. +The following steps assume: -!!! note - Physical streaming replication copies data blocks directly from the primary to the standby. If these blocks are encrypted on the primary, they arrive **encrypted** on the standby. Therefore, the standby **must** have access to the **exact same encryption keys** as the primary to decrypt and read this data. +* You have configured a global key provider for both **primary** and **standby** (see [Configure Key Management (KMS)](global-key-provider-configuration/index.md)). +* You have enabled `pg_tde` and you have setup at least one active key on the **primary**. +* Both primary and standby run the **same** Percona PostgreSQL version. -## Concepts +## 1. Configure the Primary -* **`pg_tde`:** Percona's extension for Transparent Data Encryption in PostgreSQL. -* **`heap_tde`:** The table access method provided by `pg_tde` that encrypts entire tables using a principal key. -* **principal Key:** The central key used by `pg_tde` to encrypt/decrypt data. Identified by a `pg_tde_principal_key_name`. -* **Key Provider:** The mechanism `pg_tde` uses to fetch the principal key (e.g., local files, HashiCorp Vault). Configured via `pg_tde_key_provider`. +### Configure postgresql.conf -## Set Up Key Provider and principal Key for Replication +* Ensure you set the following replication settings in `postgresql.conf`: -The standby server(s) **must** be configured to use the **same key provider settings** and access the **same principal key (by name and content)** as the primary server. + ```ini + # Example of WAL and replication settings + wal_level = replica + max_wal_senders = 5 + max_replication_slots = 1 + wal_keep_size = '1GB' + # Enable TDE in WAL pipeline + shared_preload_libraries = 'pg_tde' # Loads TDE hooks at server start. + ``` -### Scenario 1: File-based Key Provider + !!! note + Make sure you set `max_replication_slots` before creating any slot. -1. **Primary:** - * In `postgresql.conf`: +* (Optional - **To review**) Create a physical slot to retain the encrypted WAL, set `max_replication_slots ≥ 1` and then: - ```ini - pg_tde_key_provider = 'file' - pg_tde_file_keys_path = '/var/lib/postgresql/pg_tde_keys' # Example directory - pg_tde_principal_key_name = 'my_primary_principal_key' - ``` + ```sql + SELECT pg_create_physical_replication_slot('tde_slot'); + ``` - * Create the directory: `sudo -u postgres mkdir /var/lib/postgresql/pg_tde_keys` - * `sudo -u postgres chmod 700 /var/lib/postgresql/pg_tde_keys` - * Place your actual principal key file (for example, a file containing a 32-byte AES key) inside this directory, named `my_primary_principal_key`: - `sudo -u postgres dd if=/dev/urandom of=/var/lib/postgresql/pg_tde_keys/my_primary_principal_key bs=32 count=1` - `sudo -u postgres chmod 600 /var/lib/postgresql/pg_tde_keys/my_primary_principal_key` - * After restarting PostgreSQL, in `psql`: +* Ensure you have [configured the global key provider](global-key-provider-configuration/index.md). +* Create the [principal key](functions#pg_tde_set_server_key_using_global_key_provider). +* Enable [WAL encryption](wal-encryption). **Restart** PostgreSQL. +* Ensure the extension is installed in each database: - ```sql - CREATE EXTENSION IF NOT EXISTS pg_tde; -- If not already done - -- Add the key definition to pg_tde's catalog (it will read from the file) - SELECT pg_tde_principal_key_add('my_primary_principal_key', NULL, 'file'); - -- Set this key as the active principal key - SELECT pg_tde_set_principal_key('my_primary_principal_key', 'file'); - ``` + ```sql + CREATE EXTENSION IF NOT EXISTS pg_tde; + ``` -2. **Standby:** - * Before starting the standby, **securely copy** the entire `pg_tde_file_keys_path` directory and its contents (specifically the `my_primary_principal_key` file) from the primary to the exact same path on the standby (`/var/lib/postgresql/pg_tde_keys`). - * Ensure permissions on the standby's directory and key file are identical. - * In the standby's `postgresql.conf`, configure `pg_tde_key_provider`, `pg_tde_file_keys_path`, and `pg_tde_principal_key_name` to be **identical** to the primary's configuration. - * The standby, upon startup, uses these settings to load the key. +* (Optional) If you want to block any unencrypted tables, you can [enforce table-level encryption](variables#pg_tde.enforce_encryption): -### Scenario 2: HashiCorp Vault Key Provider** + ```ini + pg_tde.enforce_encryption = on + ``` -1. **Primary:** - * Ensure the principal key (e.g., `my_vault_principal_key`) exists in Vault at the configured path (for example, `secret/data/postgres/keys`). - * In `postgresql.conf`: +### Create the replication role - ```ini - pg_tde_key_provider = 'vault' - pg_tde_vault_url = 'http://your-vault-server:8200' - pg_tde_vault_token = 'your-vault-access-token' # Or use other auth methods - pg_tde_vault_mount_path = 'secret/' # Path where KV engine is mounted - # pg_tde_vault_ca_path = '/path/to/vault_ca.crt' # If using self-signed certs - pg_tde_principal_key_name = 'my_vault_principal_key' # Key name within Vault path - ``` +Ensure your primary has a replication role: - * After restarting PostgreSQL, in `psql`: +```sql +CREATE ROLE example_replicator WITH REPLICATION LOGIN PASSWORD 'example_password'; +``` - ```sql - CREATE EXTENSION IF NOT EXISTS pg_tde; - -- Add the key definition (pg_tde will fetch it from Vault) - SELECT pg_tde_principal_key_add('my_vault_principal_key', NULL, 'vault'); - -- Set this key as the active principal key - SELECT pg_tde_set_principal_key('my_vault_principal_key', 'vault'); - ``` +### Configure pg_hba.conf -2. **Standby:** - * In the standby's `postgresql.conf`, configure `pg_tde_key_provider` and all `pg_tde_vault_*` parameters, and `pg_tde_principal_key_name` to be **identical** to the primary's configuration. - * Ensure the standby server has network access to the Vault server and the necessary credentials/permissions to read the *same principal key* from Vault. - * The standby, upon startup, uses these settings to load the key. +To connect to the replication server, add the following line in `pg_hba.conf`: -**Is it possible to use the same key provider and principal key?** +```conf +host replication example_replicator standby_ip/32 scram-sha-256 +``` -**Yes, it is mandatory.** For physical replication to function with TDE, the standby *must* use the same principal key (by name and content, accessed via the same provider configuration) as the primary to decrypt the replicated data blocks. +Ensure that it is placed before the other host rules for replication and then **reload** the configuration: -## Encrypted Table Replication +```sql +SELECT pg_reload_conf(); +``` -**If a table on the primary site is encrypted, will all the standby sites also have the encrypted table replicated, given that the same key is implemented on the standby sites as well?** +## 2. Configure the Standby -Yes. +### Perform an encrypted database backup -Physical replication works at the block level. +Run the base backup from your standby machine to pull the encrypted base backup: -1. When you encrypt a table on the primary using `CREATE TABLE ... USING heap_tde;` or by setting `default_table_access_method = 'heap_tde'`, the data blocks on disk are encrypted. -2. Changes to this table (also encrypted) are written to WAL. -3. Streaming replication sends these WAL records to the standby. -4. The standby applies these WAL records, writing the encrypted blocks to its disk. -5. If the standby has the same principal key available (via identical `pg_tde` configuration and key access), it can decrypt these blocks when you query the table on the standby. +```bash +export PGPASSWORD='example_password' +pg_basebackup \ +-h primary_ip \ +-D /var/lib/pgsql/data \ +-U example_replicator \ +--wal-method=stream \ +--slot=tde_slot \ +-v -P +``` + +!!! note + Run pg_basebackup only **after** slot creation if using `--slot=tde_slot`. -If the standby lacks the correct key, replicated data remains encrypted and unreadable, and queries will fail. +### Initial standby setup -## Setup Steps +* Ensure that in `postgresql.conf` or `postgresql.auto.conf`: -This assumes the `pg_tde` extension is available for installation on both primary and standby. +```ini +shared_preload_libraries = 'pg_tde' +hot_standby = on +``` -### A. Primary Server Setup +!!! note + By default `pg_tde.inherit_global_providers = on`, so the standby inherits your global KMS configuration automatically. -1. **Install the `pg_tde` extension files.** -2. **Configure `postgresql.conf` for `pg_tde`:** +* Install the extension so the standby can register `tde_heap`: - ```ini - shared_preload_libraries = 'pg_tde' // Add pg_tde, comma-separated if others exist - - # Example for File Provider: - pg_tde_key_provider = 'file' - pg_tde_file_keys_path = '/var/lib/postgresql/pg_tde_keys' - pg_tde_principal_key_name = 'my_principal_key' - - # Example for Vault Provider: - # pg_tde_key_provider = 'vault' - # pg_tde_vault_url = 'http://your-vault-server:8200' - # pg_tde_vault_token = 's.xxxxxxxxxxxxxx' - # pg_tde_vault_mount_path = 'secret/' - # pg_tde_principal_key_name = 'my_principal_key_in_vault' - ``` +```sql +CREATE EXTENSION IF NOT EXISTS pg_tde; +``` -3. **Configure `postgresql.conf` for Replication:** +* For PostgreSQL ≥13, in `postgresql.auto.conf`, ensure you have set the following: - ```ini - wal_level = replica - max_wal_senders = 5 - archive_mode = on # Recommended - archive_command = 'cp %p /path/to/archive/%f' # Example - ``` +```ini +primary_conninfo = 'host=primary_ip port=5432 \ + user=example_replicator password=example_password \ + application_name=standby_node sslmode=verify-full \ + sslrootcert=/path/to/ca.pem' +primary_slot_name = 'tde_slot' +``` -4. **Prepare principal Key in Provider:** +### On the standby host, after pg_basebackup and configuration is set: - * **File:** Create the directory specified in `pg_tde_file_keys_path`. Create the key file inside this directory, named as per `pg_tde_principal_key_name`. +```bash +touch $PGDATA/standby.signal +``` - ```bash - sudo -u postgres mkdir -p /var/lib/postgresql/pg_tde_keys - sudo -u postgres chmod 700 /var/lib/postgresql/pg_tde_keys - sudo -u postgres dd if=/dev/urandom of=/var/lib/postgresql/pg_tde_keys/my_principal_key bs=32 count=1 - sudo -u postgres chmod 600 /var/lib/postgresql/pg_tde_keys/my_principal_key - ``` +PostgreSQL looks for the `standby.signal` (replacing `recovery.conf` as of v12) to know it should enter streaming recovery. - * **Vault:** Ensure the key exists in Vault at the correct path and is accessible. -5. **Restart PostgreSQL on Primary.** -6. **Initialize `pg_tde` and Set principal Key (in `psql`):** +!!! note + Ensure the standby has access to the **same** encryption key material or provider configuration used by the primary. - ```sql - CREATE EXTENSION pg_tde; - -- Add the key definition to pg_tde's catalog - -- For File (key_name matches pg_tde_principal_key_name and filename): - SELECT pg_tde_principal_key_add('my_principal_key', NULL, 'file'); - -- For Vault (key_name matches pg_tde_principal_key_name and key name in Vault): - -- SELECT pg_tde_principal_key_add('my_principal_key_in_vault', NULL, 'vault'); - - -- Set this key as the active principal key - -- For File: - SELECT pg_tde_set_principal_key('my_principal_key', 'file'); - -- For Vault: - -- SELECT pg_tde_set_principal_key('my_principal_key_in_vault', 'vault'); - - -- Set heap_tde as default for new tables (optional) - ALTER SYSTEM SET default_table_access_method = 'heap_tde'; - SELECT pg_reload_conf(); - ``` +## 3. Start and validate replication -7. **Create Replication User & Configure `pg_hba.conf` (standard PostgreSQL steps).** -8. **Create and Encrypt a Test Table:** +```bash +sudo systemctl start postgresql +``` - ```sql - -- If default_table_access_method = 'heap_tde' - CREATE TABLE encrypted_data (id INT PRIMARY KEY, data TEXT); - -- Or explicitly: - -- CREATE TABLE encrypted_data (id INT PRIMARY KEY, data TEXT) USING heap_tde; +* On primary: - INSERT INTO encrypted_data VALUES (1, 'Secret on primary'); - ``` +```sql +SELECT client_addr, state +FROM pg_stat_replication; +``` -### B. Standby Server Setup** - -1. **Install `pg_tde` extension files.** -2. **Configure `postgresql.conf` for `pg_tde` - IDENTICAL to Primary:** - Copy the `pg_tde_*` GUCs from the primary's `postgresql.conf` to the standby's `postgresql.conf`. Ensure they are exactly the same. -3. **Ensure principal Key Accessibility:** - * **File Provider:** Securely copy the *entire key directory* (e.g., `/var/lib/postgresql/pg_tde_keys`) and its contents from primary to the standby at the *exact same path*. Ensure permissions are identical. - * **Vault Provider:** Ensure standby has network and credential access to the *same key name* in Vault. -4. **Stop Standby PostgreSQL service (if running).** -5. **Remove any existing data from the standby's data directory.** -6. **Take a Base Backup from Primary:** - - ```bash - # Run as postgres OS user on the standby server - pg_basebackup -h your_primary_ip -U replicator_user -p 5432 -D /path/to/standby_data_dir -Fp -Xs -P -R - ``` +* On standby: -7. **Start PostgreSQL on Standby.** - The standby will connect, stream WAL, and apply changes. With identical `pg_tde` config and key access, it will decrypt data. - -## Test the Implementation - -1. **Follow all steps in Section 4 (A and B) precisely.** -2. **On the Primary:** - * Insert more data into `encrypted_data`. - - ```sql - INSERT INTO encrypted_data VALUES (2, 'More secrets from primary'); - ``` - -3. **On the Standby:** - * Wait for replication. Connect via `psql`. - * **Crucial Test:** Query the encrypted table: - - ```sql - SELECT * FROM encrypted_data; - ``` - - **Expected Result:** You should see all data, including new rows, decrypted successfully. - - ```sql - id | data - ----+------------------------- - 1 | Secret on primary - 2 | More secrets from primary - (2 rows) - ``` - - * **Negative Test (Optional):** Temporarily make the principal key file inaccessible on the standby (e.g., rename `pg_tde_file_keys_path` directory or the key file within it) or revoke Vault token. Restart standby. Queries to `encrypted_data` should now fail with a TDE key access error. Restore access and restart to confirm functionality. - -## Limitations and Important Considerations - -* **principal Key Security:** The principal key is paramount. Securely back it up. For file provider, protect the key file and its directory stringently. -* **Identical Configuration:** All `pg_tde_*` GUCs relevant to key provider and key name **must** be identical on primary and all standbys. -* **Key Rotation (`pg_tde_rotate_principal_key`):** - 1. Create/place the new key in your chosen key provider (new file in `pg_tde_file_keys_path` or new key in Vault). - 2. On the primary, add the new key to `pg_tde`'s catalog: - `SELECT pg_tde_principal_key_add('new_principal_key_name', NULL, 'your_provider');` - 3. Perform rotation: - `SELECT pg_tde_rotate_principal_key('old_principal_key_name', 'new_principal_key_name', 'your_provider');` - This re-encrypts the internal DEKs with the new principal key. Data files are re-encrypted lazily or via `pg_tde_encrypt_table()`. - 4. Update `pg_tde_principal_key_name` in `postgresql.conf` on the primary to `'new_principal_key_name'`. - 5. **Crucially for replication:** - * Ensure the `new_principal_key_name` (and its actual key material) is distributed/accessible to all standbys. - * Update `pg_tde_principal_key_name` in `postgresql.conf` on all standbys to match the primary. - * A rolling restart of standbys might be necessary after their config and key material are updated. -* **Promotion:** A standby promoted to primary will already have the necessary `pg_tde` setup and key access to function as a TDE-enabled primary. +```sql +SELECT + pg_is_in_recovery() AS in_recovery, + pg_last_wal_receive_lsn() AS receive_lsn, + pg_last_wal_replay_lsn() AS replay_lsn; +``` From c736a0d2ef3e570558f47c7d8c6e324aefa49593 Mon Sep 17 00:00:00 2001 From: Dragos Andriciuc Date: Wed, 4 Jun 2025 17:14:15 +0300 Subject: [PATCH 06/11] Update replication.md updated with added -c fast option for pg_basebackup and implemented Zsolt's feedback --- .../pg_tde/documentation/docs/replication.md | 68 +++---------------- 1 file changed, 10 insertions(+), 58 deletions(-) diff --git a/contrib/pg_tde/documentation/docs/replication.md b/contrib/pg_tde/documentation/docs/replication.md index 0066bdb6b447a..9bf05ef552b92 100644 --- a/contrib/pg_tde/documentation/docs/replication.md +++ b/contrib/pg_tde/documentation/docs/replication.md @@ -18,36 +18,20 @@ The following steps assume: # Example of WAL and replication settings wal_level = replica max_wal_senders = 5 - max_replication_slots = 1 + max_replication_slots = 10 # the default value wal_keep_size = '1GB' # Enable TDE in WAL pipeline shared_preload_libraries = 'pg_tde' # Loads TDE hooks at server start. ``` - !!! note - Make sure you set `max_replication_slots` before creating any slot. - -* (Optional - **To review**) Create a physical slot to retain the encrypted WAL, set `max_replication_slots ≥ 1` and then: - - ```sql - SELECT pg_create_physical_replication_slot('tde_slot'); - ``` - -* Ensure you have [configured the global key provider](global-key-provider-configuration/index.md). +* Ensure you have configured the provider. * Create the [principal key](functions#pg_tde_set_server_key_using_global_key_provider). -* Enable [WAL encryption](wal-encryption). **Restart** PostgreSQL. -* Ensure the extension is installed in each database: +* Ensure the extension is installed where it is needed: ```sql CREATE EXTENSION IF NOT EXISTS pg_tde; ``` -* (Optional) If you want to block any unencrypted tables, you can [enforce table-level encryption](variables#pg_tde.enforce_encryption): - - ```ini - pg_tde.enforce_encryption = on - ``` - ### Create the replication role Ensure your primary has a replication role: @@ -79,17 +63,15 @@ Run the base backup from your standby machine to pull the encrypted base backup: ```bash export PGPASSWORD='example_password' pg_basebackup \ --h primary_ip \ --D /var/lib/pgsql/data \ --U example_replicator \ ---wal-method=stream \ ---slot=tde_slot \ --v -P + -h primary_ip \ + -D /var/lib/pgsql/data \ + -U example_replicator \ + --wal-method=stream \ + --slot=tde_slot \ + -c fast \ + -v -P ``` -!!! note - Run pg_basebackup only **after** slot creation if using `--slot=tde_slot`. - ### Initial standby setup * Ensure that in `postgresql.conf` or `postgresql.auto.conf`: @@ -99,36 +81,6 @@ shared_preload_libraries = 'pg_tde' hot_standby = on ``` -!!! note - By default `pg_tde.inherit_global_providers = on`, so the standby inherits your global KMS configuration automatically. - -* Install the extension so the standby can register `tde_heap`: - -```sql -CREATE EXTENSION IF NOT EXISTS pg_tde; -``` - -* For PostgreSQL ≥13, in `postgresql.auto.conf`, ensure you have set the following: - -```ini -primary_conninfo = 'host=primary_ip port=5432 \ - user=example_replicator password=example_password \ - application_name=standby_node sslmode=verify-full \ - sslrootcert=/path/to/ca.pem' -primary_slot_name = 'tde_slot' -``` - -### On the standby host, after pg_basebackup and configuration is set: - -```bash -touch $PGDATA/standby.signal -``` - -PostgreSQL looks for the `standby.signal` (replacing `recovery.conf` as of v12) to know it should enter streaming recovery. - -!!! note - Ensure the standby has access to the **same** encryption key material or provider configuration used by the primary. - ## 3. Start and validate replication ```bash From f704afaed55b4c7a1c84df35b08b53ff8f70f136 Mon Sep 17 00:00:00 2001 From: Dragos Andriciuc Date: Thu, 5 Jun 2025 16:09:25 +0300 Subject: [PATCH 07/11] Update replication.md updated based on feedback --- .../pg_tde/documentation/docs/replication.md | 18 ++++-------------- 1 file changed, 4 insertions(+), 14 deletions(-) diff --git a/contrib/pg_tde/documentation/docs/replication.md b/contrib/pg_tde/documentation/docs/replication.md index 9bf05ef552b92..30d414ff6a8e1 100644 --- a/contrib/pg_tde/documentation/docs/replication.md +++ b/contrib/pg_tde/documentation/docs/replication.md @@ -4,26 +4,15 @@ This section outlines how to set up PostgreSQL streaming replication when the `p The following steps assume: -* You have configured a global key provider for both **primary** and **standby** (see [Configure Key Management (KMS)](global-key-provider-configuration/index.md)). * You have enabled `pg_tde` and you have setup at least one active key on the **primary**. -* Both primary and standby run the **same** Percona PostgreSQL version. +* You have configured a global key provider for the **primary**, see [Configure Key Management (KMS)](global-key-provider-configuration/index.md) for more information. +* Ensure the certificate files are accessible for the standby, and that `pg_tde` is added to the shared preload libraries. ## 1. Configure the Primary ### Configure postgresql.conf -* Ensure you set the following replication settings in `postgresql.conf`: - - ```ini - # Example of WAL and replication settings - wal_level = replica - max_wal_senders = 5 - max_replication_slots = 10 # the default value - wal_keep_size = '1GB' - # Enable TDE in WAL pipeline - shared_preload_libraries = 'pg_tde' # Loads TDE hooks at server start. - ``` - +* Ensure you have configured `postgresql.conf`. * Ensure you have configured the provider. * Create the [principal key](functions#pg_tde_set_server_key_using_global_key_provider). * Ensure the extension is installed where it is needed: @@ -68,6 +57,7 @@ pg_basebackup \ -U example_replicator \ --wal-method=stream \ --slot=tde_slot \ + -C \ -c fast \ -v -P ``` From 6bca4af9663c81a7de3298166c279a230bbc48f5 Mon Sep 17 00:00:00 2001 From: Dragos Andriciuc Date: Fri, 6 Jun 2025 12:09:15 +0300 Subject: [PATCH 08/11] Update replication.md updated with feedback, removed global for configuring key providers, reordered the setup to ensure pg_tde is enabled and THEN create a key, small clarification updates --- .../pg_tde/documentation/docs/replication.md | 22 +++++++++++-------- 1 file changed, 13 insertions(+), 9 deletions(-) diff --git a/contrib/pg_tde/documentation/docs/replication.md b/contrib/pg_tde/documentation/docs/replication.md index 30d414ff6a8e1..587a9d04f4d50 100644 --- a/contrib/pg_tde/documentation/docs/replication.md +++ b/contrib/pg_tde/documentation/docs/replication.md @@ -4,23 +4,28 @@ This section outlines how to set up PostgreSQL streaming replication when the `p The following steps assume: -* You have enabled `pg_tde` and you have setup at least one active key on the **primary**. -* You have configured a global key provider for the **primary**, see [Configure Key Management (KMS)](global-key-provider-configuration/index.md) for more information. +* You have enabled `pg_tde`. +* You have configured a key provider for the **primary**, see [Configure Key Management (KMS)](global-key-provider-configuration/index.md) for more information. * Ensure the certificate files are accessible for the standby, and that `pg_tde` is added to the shared preload libraries. ## 1. Configure the Primary ### Configure postgresql.conf -* Ensure you have configured `postgresql.conf`. -* Ensure you have configured the provider. -* Create the [principal key](functions#pg_tde_set_server_key_using_global_key_provider). +* Ensure you have configured `postgresql.conf` and that it contains the following line: + +```ini +shared_preload_libraries = 'pg_tde' +``` + * Ensure the extension is installed where it is needed: ```sql CREATE EXTENSION IF NOT EXISTS pg_tde; ``` +* Create the [principal key](functions#pg_tde_set_server_key_using_global_key_provider). + ### Create the replication role Ensure your primary has a replication role: @@ -31,7 +36,7 @@ CREATE ROLE example_replicator WITH REPLICATION LOGIN PASSWORD 'example_password ### Configure pg_hba.conf -To connect to the replication server, add the following line in `pg_hba.conf`: +To allow the replica to connect to the primary server, add the following line in `pg_hba.conf`: ```conf host replication example_replicator standby_ip/32 scram-sha-256 @@ -45,7 +50,7 @@ SELECT pg_reload_conf(); ## 2. Configure the Standby -### Perform an encrypted database backup +### Perform a database backup Run the base backup from your standby machine to pull the encrypted base backup: @@ -64,11 +69,10 @@ pg_basebackup \ ### Initial standby setup -* Ensure that in `postgresql.conf` or `postgresql.auto.conf`: +* Ensure that the following line is present in `postgresql.conf` or `postgresql.auto.conf`: ```ini shared_preload_libraries = 'pg_tde' -hot_standby = on ``` ## 3. Start and validate replication From aea50a2afc8828ccf38779b771d387940b660929 Mon Sep 17 00:00:00 2001 From: Dragos Andriciuc Date: Mon, 9 Jun 2025 17:51:19 +0300 Subject: [PATCH 09/11] Update replication.md removed considerations and added a paragraph directing the user to install and config, also added a link for tde_heap docs --- contrib/pg_tde/documentation/docs/replication.md | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/contrib/pg_tde/documentation/docs/replication.md b/contrib/pg_tde/documentation/docs/replication.md index 587a9d04f4d50..3bc11afeb2e50 100644 --- a/contrib/pg_tde/documentation/docs/replication.md +++ b/contrib/pg_tde/documentation/docs/replication.md @@ -1,12 +1,8 @@ # Streaming Replication with tde_heap -This section outlines how to set up PostgreSQL streaming replication when the `pg_tde` extension (specifically the `tde_heap` access method) is enabled on the primary server. +This section outlines how to set up PostgreSQL streaming replication when the `pg_tde` extension (specifically the [`tde_heap`](index/table-access-method.md) access method) is enabled on the primary server. -The following steps assume: - -* You have enabled `pg_tde`. -* You have configured a key provider for the **primary**, see [Configure Key Management (KMS)](global-key-provider-configuration/index.md) for more information. -* Ensure the certificate files are accessible for the standby, and that `pg_tde` is added to the shared preload libraries. +The following steps assume you have [installed](install.md) and [configured](setup.md) `pg_tde`. ## 1. Configure the Primary From 3c7e6f34269c94698a335a0d17bf5562524623fc Mon Sep 17 00:00:00 2001 From: Dragos Andriciuc Date: Wed, 11 Jun 2025 17:12:02 +0300 Subject: [PATCH 10/11] Update replication.md Updated the following: - Reworded the second paragraph and removed the install.md reference. I also added (which includes...) to ensure that experienced users don't just ignore the setup link and have a proper quick view of the prerequisites as well as helping new users to reinforce what they need to have configured already. - Removed CREATE EXTENSION from step 1 Configure the Primary, it is redundant. I also updated the principal key function step with a fixed link - Updated primary replication role paragraph - Removed initial standby setup as we already cover this - Updated step 3 Start and Validate replication with an intro text and added a tip to remind the user they can verify if the encrypt is --- .../pg_tde/documentation/docs/replication.md | 38 ++++++++----------- 1 file changed, 15 insertions(+), 23 deletions(-) diff --git a/contrib/pg_tde/documentation/docs/replication.md b/contrib/pg_tde/documentation/docs/replication.md index 3bc11afeb2e50..11d877a9ea37f 100644 --- a/contrib/pg_tde/documentation/docs/replication.md +++ b/contrib/pg_tde/documentation/docs/replication.md @@ -2,29 +2,24 @@ This section outlines how to set up PostgreSQL streaming replication when the `pg_tde` extension (specifically the [`tde_heap`](index/table-access-method.md) access method) is enabled on the primary server. -The following steps assume you have [installed](install.md) and [configured](setup.md) `pg_tde`. +Before you begin, ensure you have followed the [`pg_tde` setup instructions](setup.md), which includes: -## 1. Configure the Primary - -### Configure postgresql.conf - -* Ensure you have configured `postgresql.conf` and that it contains the following line: +- Installing the `pg_tde` extension binaries, where they are needed, on **both** the primary and standby servers. +- Configuring `shared_preload_libraries = 'pg_tde'` in `postgresql.conf` on **both** systems. +- Initializing the extension and setting a principal key on the **primary**. -```ini -shared_preload_libraries = 'pg_tde' -``` +!!! note + You do **not** need to run `CREATE EXTENSION` on the standby. It will be replicated automatically. -* Ensure the extension is installed where it is needed: +## 1. Configure the Primary - ```sql - CREATE EXTENSION IF NOT EXISTS pg_tde; - ``` +### Create a principal key -* Create the [principal key](functions#pg_tde_set_server_key_using_global_key_provider). +Use the [`pg_tde_set_server_key_using_global_key_provider`](functions.md#pg_tde_set_server_key_using_global_key_provider) function to create a principal key. ### Create the replication role -Ensure your primary has a replication role: +Create a replication role on the primary: ```sql CREATE ROLE example_replicator WITH REPLICATION LOGIN PASSWORD 'example_password'; @@ -63,16 +58,10 @@ pg_basebackup \ -v -P ``` -### Initial standby setup - -* Ensure that the following line is present in `postgresql.conf` or `postgresql.auto.conf`: - -```ini -shared_preload_libraries = 'pg_tde' -``` - ## 3. Start and validate replication +Start the PostgreSQL service: + ```bash sudo systemctl start postgresql ``` @@ -92,3 +81,6 @@ SELECT pg_last_wal_receive_lsn() AS receive_lsn, pg_last_wal_replay_lsn() AS replay_lsn; ``` + +!!! tip + Want to verify if everything works? Run `SELECT pg_tde_is_encrypted('your_encrypted_table');` on the standby to confirm that the encryption is active and the keys are resolved. From 23189d7954b87410f26a461bebfab97743c626e7 Mon Sep 17 00:00:00 2001 From: Dragos Andriciuc Date: Wed, 25 Jun 2025 12:05:23 +0300 Subject: [PATCH 11/11] updated based on latest feedback - updated intro to remove () and replace with , , for tde heap access method mention - removed assumption steps when talking about pg_tde setup - added Configure postgresql.conf step for configuring the standby after db backup - updated step 3 with clear assumption that primary and standby run separately and added a warning regarding Key Management consistency between replica and primary - updated tip to include a way to copy the command with one click and added that after creating an encrypted table you can use it --- .../pg_tde/documentation/docs/replication.md | 33 ++++++++++++++----- 1 file changed, 25 insertions(+), 8 deletions(-) diff --git a/contrib/pg_tde/documentation/docs/replication.md b/contrib/pg_tde/documentation/docs/replication.md index 11d877a9ea37f..88af22ba04dde 100644 --- a/contrib/pg_tde/documentation/docs/replication.md +++ b/contrib/pg_tde/documentation/docs/replication.md @@ -1,12 +1,8 @@ # Streaming Replication with tde_heap -This section outlines how to set up PostgreSQL streaming replication when the `pg_tde` extension (specifically the [`tde_heap`](index/table-access-method.md) access method) is enabled on the primary server. +This section outlines how to set up PostgreSQL streaming replication when the `pg_tde` extension, specifically the [`tde_heap`](index/table-access-method.md) access method, is enabled on the primary server. -Before you begin, ensure you have followed the [`pg_tde` setup instructions](setup.md), which includes: - -- Installing the `pg_tde` extension binaries, where they are needed, on **both** the primary and standby servers. -- Configuring `shared_preload_libraries = 'pg_tde'` in `postgresql.conf` on **both** systems. -- Initializing the extension and setting a principal key on the **primary**. +Before you begin, ensure you have followed the [`pg_tde` setup instructions](setup.md). !!! note You do **not** need to run `CREATE EXTENSION` on the standby. It will be replicated automatically. @@ -58,14 +54,31 @@ pg_basebackup \ -v -P ``` +### Configure postgresql.conf + +After the base backup completes, add the following line to the standby's `postgresql.conf` file: + +```ini +shared_preload_libraries = 'pg_tde' +``` + ## 3. Start and validate replication -Start the PostgreSQL service: +Assuming that the primary and the standby are running on separate hosts, start the PostgreSQL service: ```bash sudo systemctl start postgresql ``` +!!! warning "Key management consistency **required** for replication" + + If you're using a KMS provider, such as Vault or KMIP, make sure that both the primary and the standby have access to the **same** key management configuration, and that the paths to the configuration files are identical on both systems. + + For example: + + - If you configure Vault with a secret path: `/path/to/secret.file`, then that file **must** exist at the same path on both the primary and the standby. + - If you use the `keyring_file` provider, be aware that it stores key material in a local file and it is **not designed** for shared or concurrent use across multiple servers. It is **not recommended** in replication setups. + * On primary: ```sql @@ -83,4 +96,8 @@ SELECT ``` !!! tip - Want to verify if everything works? Run `SELECT pg_tde_is_encrypted('your_encrypted_table');` on the standby to confirm that the encryption is active and the keys are resolved. + Want to verify that everything is working? After creating an encrypted table on the primary, run the following command on the standby to confirm that the encryption is active and the keys are resolved: + + ```sql + SELECT pg_tde_is_encrypted('your_encrypted_table'); + ```