From 265799a8ce1759030cb31ba490270c102e1d3746 Mon Sep 17 00:00:00 2001 From: Dragos Andriciuc Date: Tue, 5 Aug 2025 12:15:26 +0300 Subject: [PATCH 01/15] Add WAL content for 2.0 release --- .../pg_tde/documentation/docs/architecture/architecture.md | 5 +---- .../documentation/docs/command-line-tools/pg-waldump.md | 3 --- contrib/pg_tde/documentation/docs/faq.md | 5 +---- contrib/pg_tde/documentation/docs/functions.md | 3 --- contrib/pg_tde/documentation/docs/index/tde-limitations.md | 1 - contrib/pg_tde/documentation/docs/test.md | 2 +- contrib/pg_tde/documentation/docs/wal-encryption.md | 7 +++---- contrib/pg_tde/documentation/mkdocs.yml | 2 +- 8 files changed, 7 insertions(+), 21 deletions(-) diff --git a/contrib/pg_tde/documentation/docs/architecture/architecture.md b/contrib/pg_tde/documentation/docs/architecture/architecture.md index bc1df2f25df6e..2902d1553898e 100644 --- a/contrib/pg_tde/documentation/docs/architecture/architecture.md +++ b/contrib/pg_tde/documentation/docs/architecture/architecture.md @@ -20,7 +20,7 @@ The following sections break down the key architectural components of this desig * Indexes * Sequences * Temporary tables -* Write Ahead Log (WAL), still in beta. **Do not enable this feature in production environments**. +* Write Ahead Log (WAL) **Extension** means that `pg_tde` should be implemented only as an extension, possibly compatible with any PostgreSQL distribution, including the open source community version. This requires changes in the PostgreSQL core to make it more extensible. Therefore, `pg_tde` currently works only with the [Percona Server for PostgreSQL](https://docs.percona.com/postgresql/17/index.html) - a binary replacement of community PostgreSQL and included in Percona Distribution for PostgreSQL. @@ -82,9 +82,6 @@ Later decisions are made using a slightly modified Storage Manager (SMGR) API: w ### WAL encryption -!!! note - The WAL encryption feature is currently in beta and is not effective unless explicitly enabled. It is not yet production ready. **Do not enable this feature in production environments**. - WAL encryption is controlled globally via a global GUC variable, `pg_tde.wal_encrypt`, that requires a server restart. WAL keys also contain the [LSN](https://www.postgresql.org/docs/17/wal-internals.html) of the first WAL write after key creation. This allows `pg_tde` to know which WAL ranges are encrypted or not and with which key. diff --git a/contrib/pg_tde/documentation/docs/command-line-tools/pg-waldump.md b/contrib/pg_tde/documentation/docs/command-line-tools/pg-waldump.md index 0362eadcef7f0..a3ac50df4df02 100644 --- a/contrib/pg_tde/documentation/docs/command-line-tools/pg-waldump.md +++ b/contrib/pg_tde/documentation/docs/command-line-tools/pg-waldump.md @@ -2,9 +2,6 @@ [`pg_waldump` :octicons-link-external-16:](https://www.postgresql.org/docs/current/pgwaldump.html) is a tool to display a human-readable rendering of the Write-Ahead Log (WAL) of a PostgreSQL database cluster. -!!! warning - The WAL encryption feature is currently in beta and is not effective unless explicitly enabled. It is not yet production ready. **Do not enable this feature in production environments**. - To read encrypted WAL records, `pg_waldump` supports the following additional arguments: * `keyring_path` is the directory where the keyring configuration files for WAL are stored. The following files are included: diff --git a/contrib/pg_tde/documentation/docs/faq.md b/contrib/pg_tde/documentation/docs/faq.md index b244818371144..9e9c88f181a65 100644 --- a/contrib/pg_tde/documentation/docs/faq.md +++ b/contrib/pg_tde/documentation/docs/faq.md @@ -94,9 +94,6 @@ The principal key is used to encrypt the internal keys. The principal key is sto ### WAL encryption -!!! note - WAL encryption is currently in beta and is not effective unless explicitly enabled. It is not yet production ready. **Do not enable this feature in production environments**. - WAL encryption is done globally for the entire database cluster. All modifications to any database within a PostgreSQL cluster are written to the same WAL to maintain data consistency and integrity and ensure that PostgreSQL cluster can be restored to a consistent state. Therefore, WAL is encrypted globally. When you turn on WAL encryption, `pg_tde` encrypts entire WAL files starting from the first WAL write after the server was started with the encryption turned on. @@ -140,7 +137,7 @@ Since the `SET ACCESS METHOD` command drops hint bits and this may affect the pe You must restart the database in the following cases to apply the changes: * after you enabled the `pg_tde` extension -* when enabling WAL encryption, which is currently in beta. **Do not enable this feature in production environments**. +* when enabling WAL encryption After that, no database restart is required. When you create or alter the table using the `tde_heap` access method, the files are marked as those that require encryption. The encryption happens at the storage manager level, before a transaction is written to disk. Read more about [how tde_heap works](index/table-access-method.md#how-tde_heap-works-with-pg_tde). diff --git a/contrib/pg_tde/documentation/docs/functions.md b/contrib/pg_tde/documentation/docs/functions.md index 73f2da208faa0..9ef29afcc44fc 100644 --- a/contrib/pg_tde/documentation/docs/functions.md +++ b/contrib/pg_tde/documentation/docs/functions.md @@ -283,9 +283,6 @@ SELECT pg_tde_set_server_key_using_global_key_provider( ); ``` -!!! warning - The WAL encryption feature is currently in beta and is not effective unless explicitly enabled. It is not yet production ready. **Do not enable this feature in production environments**. -======= The `ensure_new_key` parameter instructs the function how to handle a principal key during key rotation: * If set to `true`, a new key must be unique. diff --git a/contrib/pg_tde/documentation/docs/index/tde-limitations.md b/contrib/pg_tde/documentation/docs/index/tde-limitations.md index 8e3376671aea7..10c1627b789ce 100644 --- a/contrib/pg_tde/documentation/docs/index/tde-limitations.md +++ b/contrib/pg_tde/documentation/docs/index/tde-limitations.md @@ -3,6 +3,5 @@ The following are current limitations of `pg_tde`: * System tables, which include statistics data and database statistics, are currently **not encrypted**. -* The WAL encryption feature is currently in beta and is not effective unless explicitly enabled. It is not yet production ready. **Do not enable this feature in production environments**. [View the versions and supported deployments :material-arrow-right:](supported-versions.md){.md-button} diff --git a/contrib/pg_tde/documentation/docs/test.md b/contrib/pg_tde/documentation/docs/test.md index be73e73863e76..e4578e6bf6ed3 100644 --- a/contrib/pg_tde/documentation/docs/test.md +++ b/contrib/pg_tde/documentation/docs/test.md @@ -59,4 +59,4 @@ ALTER TABLE table_name SET ACCESS METHOD tde_heap; ## Next steps -[Configure WAL Encryption (tech preview) :material-arrow-right:](wal-encryption.md){.md-button} +[Configure WAL encryption :material-arrow-right:](wal-encryption.md){.md-button} diff --git a/contrib/pg_tde/documentation/docs/wal-encryption.md b/contrib/pg_tde/documentation/docs/wal-encryption.md index 33547d0e359c4..6de83230c11de 100644 --- a/contrib/pg_tde/documentation/docs/wal-encryption.md +++ b/contrib/pg_tde/documentation/docs/wal-encryption.md @@ -1,7 +1,4 @@ -# Configure WAL Encryption (tech preview) - -!!! warning - The WAL encryption feature is currently in beta and is not effective unless explicitly enabled. It is not yet production ready. **Do not enable this feature in production environments**. +# Configure WAL encryption Before enabling WAL encryption, follow the steps below to create a principal key and configure it for WAL: @@ -118,3 +115,5 @@ Now WAL files start to be encrypted for both encrypted and unencrypted tables. For more technical references related to architecture, variables or functions, see: [Technical Reference](advanced-topics/tech-reference.md){.md-button} + +💬 Need help customizing this for your infrastructure? [Contact Percona support :octicons-link-external-16:](get-help.md) \ No newline at end of file diff --git a/contrib/pg_tde/documentation/mkdocs.yml b/contrib/pg_tde/documentation/mkdocs.yml index 43725bb5de5e6..bc97c267b4777 100644 --- a/contrib/pg_tde/documentation/mkdocs.yml +++ b/contrib/pg_tde/documentation/mkdocs.yml @@ -185,7 +185,7 @@ nav: - "Keyring file configuration": global-key-provider-configuration/keyring.md - "2.2 Global Principal Key configuration": global-key-provider-configuration/set-principal-key.md - "3. Validate encryption with pg_tde": test.md - - "4. Configure WAL encryption (tech preview)": wal-encryption.md + - "4. Configure WAL encryption": wal-encryption.md - "Technical reference": - "Overview": advanced-topics/tech-reference.md - "Architecture": architecture/architecture.md From c150be42db0a86afb6c98cf955b0459179cf8c36 Mon Sep 17 00:00:00 2001 From: Dragos Andriciuc Date: Thu, 7 Aug 2025 15:09:51 +0300 Subject: [PATCH 02/15] improve functions.md with small text fix and reword introductory paragraph for Global Principal key config - fix a small space in functions and add * to pg_tde_set_ - reworded introductory paragraph in Global Principal Key configuration, added descriptions for the two functions used to config the global principal key - added explanatory paragraph to Next steps for clarity --- contrib/pg_tde/documentation/docs/functions.md | 2 +- .../set-principal-key.md | 12 +++++++++++- 2 files changed, 12 insertions(+), 2 deletions(-) diff --git a/contrib/pg_tde/documentation/docs/functions.md b/contrib/pg_tde/documentation/docs/functions.md index 9ef29afcc44fc..cd1241bcdd0d8 100644 --- a/contrib/pg_tde/documentation/docs/functions.md +++ b/contrib/pg_tde/documentation/docs/functions.md @@ -240,7 +240,7 @@ SELECT pg_tde_create_key_using_database_key_provider( ### pg_tde_create_key_using_global_key_provider -Creates a principal key at a global key provider with the given name. Use this key later with the `pg_tde_set_` series of functions. +Creates a principal key at a global key provider with the given name. Use this key later with the `pg_tde_set_*` series of functions. ```sql SELECT pg_tde_create_key_using_global_key_provider( diff --git a/contrib/pg_tde/documentation/docs/global-key-provider-configuration/set-principal-key.md b/contrib/pg_tde/documentation/docs/global-key-provider-configuration/set-principal-key.md index 1d4e55788d5df..781ce5869ddba 100644 --- a/contrib/pg_tde/documentation/docs/global-key-provider-configuration/set-principal-key.md +++ b/contrib/pg_tde/documentation/docs/global-key-provider-configuration/set-principal-key.md @@ -1,10 +1,16 @@ # Global Principal Key configuration -You can configure a default principal key using a global key provider. This key will be used by all databases that do not have their own encryption keys configured. The function **both** sets the principal key and rotates internal keys as needed. +You can configure a default principal key using a global key provider. This key is used by all databases that do not have their own encryption keys configured. + +There are two main functions for this: + +- [pg_tde_create_key_using_global_key_provider()](../functions.md#pg_tde_create_key_using_global_key_provider) creates a principal key at a global key provider +- [pg_tde_set_default_key_using_global_key_provider()](../functions.md#pg_tde_set_default_key_using_global_key_provider) sets the default principal key and rotates the internal encryption key if one is already configured ## Create a default principal key !!! note + The sample output below is for demonstration purposes only. Be sure to replace the key name and provider with your actual values. To create a global principal key, run: @@ -57,6 +63,7 @@ SELECT pg_tde_set_default_key_using_global_key_provider( * `global_vault_provider` is the name of the global key provider you previously configured. !!! note + If no error is reported, the action completed successfully. ## How key generation works @@ -64,8 +71,11 @@ SELECT pg_tde_set_default_key_using_global_key_provider( The key material (actual cryptographic key) is auto-generated by `pg_tde` and stored securely by the configured provider. !!! note + This process sets the **default principal key for the entire server**. Any database without a key explicitly configured will fall back to this key. ## Next steps +To confirm that encryption is working as expected, follow the validation steps: + [Validate Encryption with pg_tde :material-arrow-right:](../test.md){.md-button} From 1c12c70064064c98b8be190a17bd1d708ff3aaa9 Mon Sep 17 00:00:00 2001 From: Dragos Andriciuc Date: Thu, 14 Aug 2025 17:53:58 +0300 Subject: [PATCH 03/15] add WAL tool support to limitations, improve flow, add button to setup --- .../docs/index/tde-limitations.md | 26 +++++++++++++++++-- 1 file changed, 24 insertions(+), 2 deletions(-) diff --git a/contrib/pg_tde/documentation/docs/index/tde-limitations.md b/contrib/pg_tde/documentation/docs/index/tde-limitations.md index 10c1627b789ce..902715bc61ca8 100644 --- a/contrib/pg_tde/documentation/docs/index/tde-limitations.md +++ b/contrib/pg_tde/documentation/docs/index/tde-limitations.md @@ -1,7 +1,29 @@ # Limitations of pg_tde -The following are current limitations of `pg_tde`: +The following limitations apply to the current release: -* System tables, which include statistics data and database statistics, are currently **not encrypted**. +* PostgreSQL’s internal system tables, which include statistics and metadata, are not encrypted. +* `pg_createsubscriber` is not supported. + +## WAL encryption tool support + +The following tools and extensions in Percona Distribution for PostgreSQL have been tested and verified to work with `pg_tde` WAL encryption: + +??? note "Click to expand" + * Patroni + * `pg_basebackup` (with `--wal-method=stream`) + * `pg_resetwal` + * `pg_rewind` + * `pg_upgrade` + * `pg_waldump` + * pgBackRest + +## Next steps + +Check which PostgreSQL versions and deployment types are compatible with `pg_tde` before planning your installation. [View the versions and supported deployments :material-arrow-right:](supported-versions.md){.md-button} + +Begin the installation process when you’re ready to set up encryption. + +[Start installing `pg_tde`](../install.md){.md-button} From 5d4fe68c1aedd98b9bad68adf4bb2b3769bc39f9 Mon Sep 17 00:00:00 2001 From: Dragos Andriciuc Date: Mon, 18 Aug 2025 13:32:39 +0300 Subject: [PATCH 04/15] Update contrib/pg_tde/documentation/docs/index/tde-limitations.md Co-authored-by: Anastasia Alexandrova --- contrib/pg_tde/documentation/docs/index/tde-limitations.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/contrib/pg_tde/documentation/docs/index/tde-limitations.md b/contrib/pg_tde/documentation/docs/index/tde-limitations.md index 902715bc61ca8..1283a057c20f5 100644 --- a/contrib/pg_tde/documentation/docs/index/tde-limitations.md +++ b/contrib/pg_tde/documentation/docs/index/tde-limitations.md @@ -24,6 +24,6 @@ Check which PostgreSQL versions and deployment types are compatible with `pg_tde [View the versions and supported deployments :material-arrow-right:](supported-versions.md){.md-button} -Begin the installation process when you’re ready to set up encryption. +Begin the installation process when you're ready to set up encryption. [Start installing `pg_tde`](../install.md){.md-button} From 8769d25ab674328a2baf97935887e0519298e1c7 Mon Sep 17 00:00:00 2001 From: Dragos Andriciuc Date: Mon, 18 Aug 2025 13:33:05 +0300 Subject: [PATCH 05/15] update limitations with feedback --- contrib/pg_tde/documentation/docs/index/tde-limitations.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/contrib/pg_tde/documentation/docs/index/tde-limitations.md b/contrib/pg_tde/documentation/docs/index/tde-limitations.md index 902715bc61ca8..62f1c7eedf066 100644 --- a/contrib/pg_tde/documentation/docs/index/tde-limitations.md +++ b/contrib/pg_tde/documentation/docs/index/tde-limitations.md @@ -1,6 +1,6 @@ # Limitations of pg_tde -The following limitations apply to the current release: +Limitations of `pg_tde` (applies to release {{release}}): * PostgreSQL’s internal system tables, which include statistics and metadata, are not encrypted. * `pg_createsubscriber` is not supported. @@ -16,7 +16,7 @@ The following tools and extensions in Percona Distribution for PostgreSQL have b * `pg_rewind` * `pg_upgrade` * `pg_waldump` - * pgBackRest + * `pgBackRest` ## Next steps From 3dc94d34b2b0b1cb47920a49c41250b2582fb130 Mon Sep 17 00:00:00 2001 From: Dragos Andriciuc Date: Tue, 19 Aug 2025 14:45:38 +0300 Subject: [PATCH 06/15] add limitation regarding WAL shipping standy not supported with WAL encryption --- contrib/pg_tde/documentation/docs/index/tde-limitations.md | 1 + 1 file changed, 1 insertion(+) diff --git a/contrib/pg_tde/documentation/docs/index/tde-limitations.md b/contrib/pg_tde/documentation/docs/index/tde-limitations.md index 8b4c643a56e63..6d42832e407f5 100644 --- a/contrib/pg_tde/documentation/docs/index/tde-limitations.md +++ b/contrib/pg_tde/documentation/docs/index/tde-limitations.md @@ -4,6 +4,7 @@ Limitations of `pg_tde` (applies to release {{release}}): * PostgreSQL’s internal system tables, which include statistics and metadata, are not encrypted. * `pg_createsubscriber` is not supported. +* WAL shipping standby (warm standby) is not supported with WAL encryption. You can instead use [Streaming Replication with `tde_heap`](../replication.md), which is supported with encrypted WAL. ## WAL encryption tool support From 09b38f43f73e113ced8b008bfd5a34f600664e6d Mon Sep 17 00:00:00 2001 From: Dragos Andriciuc Date: Tue, 19 Aug 2025 15:43:34 +0300 Subject: [PATCH 07/15] remove confusing text from how tde works, link in note to KMS config for user * add table for user to select preferred KMS config --- .../global-key-provider-configuration/overview.md | 15 ++++++++++----- .../documentation/docs/index/how-does-tde-work.md | 10 ++-------- 2 files changed, 12 insertions(+), 13 deletions(-) diff --git a/contrib/pg_tde/documentation/docs/global-key-provider-configuration/overview.md b/contrib/pg_tde/documentation/docs/global-key-provider-configuration/overview.md index 800f505925715..cfe42b2488e37 100644 --- a/contrib/pg_tde/documentation/docs/global-key-provider-configuration/overview.md +++ b/contrib/pg_tde/documentation/docs/global-key-provider-configuration/overview.md @@ -9,10 +9,15 @@ To use an external KMS with `pg_tde`, follow these two steps: 2. Set the [Global Principal Key](set-principal-key.md) !!! note - While keyfiles may be acceptable for **local** or **testing environments**, KMS integration is the recommended approach for production deployments. + While key files may be acceptable for **local** or **testing environments**, KMS integration is the recommended approach for production deployments. -Select your prefered configuration from the links below: +Select your preferred configuration from the following table: -[KMIP Configuration :material-arrow-right:](kmip-server.md){.md-button} -[Vault Configuration :material-arrow-right:](vault.md){.md-button} -[Keyring File Configuration (not recommended) :material-arrow-right:](keyring.md){.md-button} +| KMS Provider | Description | Documentation | +|--------------------|-------------------------------------------------------|---------------| +| **KMIP** | Standard Key Management Interoperability Protocol. | [Configure KMIP →](kmip-server.md) | +| **Vault** | HashiCorp Vault integration (KV v2 API, KMIP engine). | [Configure Vault →](vault.md) | +| **Fortanix** | Fortanix DSM key management. | [Configure Fortanix →](kmip-fortanix.md) | +| **Thales** | Thales CipherTrust Manager and DSM. | [Configure Thales →](kmip-thales.md) | +| **OpenBao** | Community fork of Vault, supporting KV v2. | [Configure OpenBao →](kmip-openbao.md) | +| **Keyring file** *(not recommended)* | Local key file for dev/test only. | [Configure keyring file →](keyring.md) | diff --git a/contrib/pg_tde/documentation/docs/index/how-does-tde-work.md b/contrib/pg_tde/documentation/docs/index/how-does-tde-work.md index 8bce6c474c017..64e422e485633 100644 --- a/contrib/pg_tde/documentation/docs/index/how-does-tde-work.md +++ b/contrib/pg_tde/documentation/docs/index/how-does-tde-work.md @@ -6,15 +6,9 @@ To encrypt the data, two types of keys are used: * The **principal key** to encrypt database keys. It is kept separately from the database keys and is managed externally in the key management store. !!! note + For more information on managing and storing principal keys externally, including supported key management systems and the local keyring option, see [Key management overview](../global-key-provider-configuration/overview.md). - For more information on managing and storing principal keys externally, see [Configure Global Key Provider](../global-key-provider-configuration/overview.md). - -You have the following options to store and manage principal keys externally: - -* Use the HashiCorp Vault server. Only the back end KV Secrets Engine - Version 2 (API) is supported. -* Use the KMIP-compatible server. `pg_tde` has been tested with the [PyKMIP](https://pykmip.readthedocs.io/en/latest/server.html) server and [the HashiCorp Vault Enterprise KMIP Secrets Engine](https://www.vaultproject.io/docs/secrets/kmip). - -The encryption process is the following: +The encryption process works as follows: ![image](../_images/tde-flow.png) From aa604d0e8e9fa57b3a6acfc3cecee11a5d17707d Mon Sep 17 00:00:00 2001 From: Dragos Andriciuc Date: Tue, 19 Aug 2025 16:09:47 +0300 Subject: [PATCH 08/15] add mention of open source and enterprise ed being supported for pg_tde --- .../docs/global-key-provider-configuration/vault.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/contrib/pg_tde/documentation/docs/global-key-provider-configuration/vault.md b/contrib/pg_tde/documentation/docs/global-key-provider-configuration/vault.md index 00bac8074d457..ee913bc991d6d 100644 --- a/contrib/pg_tde/documentation/docs/global-key-provider-configuration/vault.md +++ b/contrib/pg_tde/documentation/docs/global-key-provider-configuration/vault.md @@ -1,6 +1,6 @@ # Vault configuration -You can configure `pg_tde` to use HashiCorp Vault as a global key provider for managing encryption keys securely. +You can configure `pg_tde` to use HashiCorp Vault as a global key provider for managing encryption keys securely. Both the open source and enterprise editions are supported. !!! note This guide assumes that your Vault server is already set up and accessible. Vault configuration is outside the scope of this document, see [Vault's official documentation](https://developer.hashicorp.com/vault/docs) for more information. From b3b7e89b1fcf7c8808c9a7d634bfeaed7efecace Mon Sep 17 00:00:00 2001 From: Dragos Andriciuc Date: Wed, 20 Aug 2025 10:58:39 +0300 Subject: [PATCH 09/15] Update the Features topic buttons for better clarity (#508) --- contrib/pg_tde/documentation/docs/features.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/contrib/pg_tde/documentation/docs/features.md b/contrib/pg_tde/documentation/docs/features.md index aa8703fc42bca..3f51c25aceaac 100644 --- a/contrib/pg_tde/documentation/docs/features.md +++ b/contrib/pg_tde/documentation/docs/features.md @@ -19,4 +19,8 @@ The following features are available for the extension: * Table-level granularity for encryption and access control * Multiple [Key management options](global-key-provider-configuration/index.md) -[Learn more about TDE and pg_tde :material-arrow-right:](index/about-tde.md){.md-button} [Get started with installation :material-arrow-right:](install.md){.md-button} +## Next steps + +Learn more about how `pg_tde` implements Transparent Data Encryption: + +[About Transparent Data Encryption :material-arrow-right:](index/about-tde.md){.md-button} From 86d8d0d420e11553151dc9d609d297b91d733d81 Mon Sep 17 00:00:00 2001 From: Dragos Andriciuc Date: Wed, 20 Aug 2025 15:16:05 +0300 Subject: [PATCH 10/15] modify limitations and global key config with feedback --- .../docs/global-key-provider-configuration/overview.md | 2 +- contrib/pg_tde/documentation/docs/index/tde-limitations.md | 5 ++--- 2 files changed, 3 insertions(+), 4 deletions(-) diff --git a/contrib/pg_tde/documentation/docs/global-key-provider-configuration/overview.md b/contrib/pg_tde/documentation/docs/global-key-provider-configuration/overview.md index cfe42b2488e37..0a43179fde400 100644 --- a/contrib/pg_tde/documentation/docs/global-key-provider-configuration/overview.md +++ b/contrib/pg_tde/documentation/docs/global-key-provider-configuration/overview.md @@ -11,7 +11,7 @@ To use an external KMS with `pg_tde`, follow these two steps: !!! note While key files may be acceptable for **local** or **testing environments**, KMS integration is the recommended approach for production deployments. -Select your preferred configuration from the following table: +`pg_tde` has been tested and approved with the following key providers: | KMS Provider | Description | Documentation | |--------------------|-------------------------------------------------------|---------------| diff --git a/contrib/pg_tde/documentation/docs/index/tde-limitations.md b/contrib/pg_tde/documentation/docs/index/tde-limitations.md index 6d42832e407f5..bc268f634ceeb 100644 --- a/contrib/pg_tde/documentation/docs/index/tde-limitations.md +++ b/contrib/pg_tde/documentation/docs/index/tde-limitations.md @@ -1,10 +1,9 @@ # Limitations of pg_tde -Limitations of `pg_tde` (applies to release {{release}}): +Limitations of `pg_tde` {{release}}: * PostgreSQL’s internal system tables, which include statistics and metadata, are not encrypted. * `pg_createsubscriber` is not supported. -* WAL shipping standby (warm standby) is not supported with WAL encryption. You can instead use [Streaming Replication with `tde_heap`](../replication.md), which is supported with encrypted WAL. ## WAL encryption tool support @@ -17,7 +16,7 @@ The following tools and extensions in Percona Distribution for PostgreSQL have b * `pg_rewind` * `pg_upgrade` * `pg_waldump` - * `pgBackRest` + * pgBackRest ## Next steps From e6bf5a9b086c7b85631ead1096e56e0c8b8b8f58 Mon Sep 17 00:00:00 2001 From: Dragos Andriciuc Date: Thu, 21 Aug 2025 13:29:56 +0300 Subject: [PATCH 11/15] add none method to basebackup and link to topic --- contrib/pg_tde/documentation/docs/index/tde-limitations.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/contrib/pg_tde/documentation/docs/index/tde-limitations.md b/contrib/pg_tde/documentation/docs/index/tde-limitations.md index bc268f634ceeb..9d9a83a636bd8 100644 --- a/contrib/pg_tde/documentation/docs/index/tde-limitations.md +++ b/contrib/pg_tde/documentation/docs/index/tde-limitations.md @@ -4,6 +4,7 @@ Limitations of `pg_tde` {{release}}: * PostgreSQL’s internal system tables, which include statistics and metadata, are not encrypted. * `pg_createsubscriber` is not supported. +* Temporary files created when queries exceed `work_mem` are not encrypted. These files may persist if the query runs for a long time or the server crashes which can expose sensitive data in plaintext on disk. ## WAL encryption tool support @@ -11,7 +12,7 @@ The following tools and extensions in Percona Distribution for PostgreSQL have b ??? note "Click to expand" * Patroni - * `pg_basebackup` (with `--wal-method=stream`) + * `pg_basebackup` (with `--wal-method=stream` or `--wal-method=none`), for details on using `pg_basebackup` with WAL encryption, see [Backup with WAL encryption enabled](../how-to/backup-wal-enabled.md) * `pg_resetwal` * `pg_rewind` * `pg_upgrade` From a551004d9f4df34731a2ac50d901922dffd9297b Mon Sep 17 00:00:00 2001 From: Dragos Andriciuc Date: Thu, 21 Aug 2025 14:55:32 +0300 Subject: [PATCH 12/15] rename WAL tool support to compatibility Disliked how in Limitations we put tool support so I rewrote the title to better reflect the limitation chapter scope --- contrib/pg_tde/documentation/docs/index/tde-limitations.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/contrib/pg_tde/documentation/docs/index/tde-limitations.md b/contrib/pg_tde/documentation/docs/index/tde-limitations.md index 9d9a83a636bd8..d5ce6928d0955 100644 --- a/contrib/pg_tde/documentation/docs/index/tde-limitations.md +++ b/contrib/pg_tde/documentation/docs/index/tde-limitations.md @@ -6,7 +6,7 @@ Limitations of `pg_tde` {{release}}: * `pg_createsubscriber` is not supported. * Temporary files created when queries exceed `work_mem` are not encrypted. These files may persist if the query runs for a long time or the server crashes which can expose sensitive data in plaintext on disk. -## WAL encryption tool support +## WAL tool compatibility (limited support) The following tools and extensions in Percona Distribution for PostgreSQL have been tested and verified to work with `pg_tde` WAL encryption: From 37fa8b21adb8aeb3a082e9d42260e323184b4ea3 Mon Sep 17 00:00:00 2001 From: Dragos Andriciuc Date: Thu, 21 Aug 2025 15:31:01 +0300 Subject: [PATCH 13/15] add Example Patroni configuration for Patroni tool This ensures that the user has a configuration example for Patroni, at least offered by us. --- .../docs/index/tde-limitations.md | 81 ++++++++++++++++--- 1 file changed, 71 insertions(+), 10 deletions(-) diff --git a/contrib/pg_tde/documentation/docs/index/tde-limitations.md b/contrib/pg_tde/documentation/docs/index/tde-limitations.md index d5ce6928d0955..b988347f028fa 100644 --- a/contrib/pg_tde/documentation/docs/index/tde-limitations.md +++ b/contrib/pg_tde/documentation/docs/index/tde-limitations.md @@ -4,20 +4,81 @@ Limitations of `pg_tde` {{release}}: * PostgreSQL’s internal system tables, which include statistics and metadata, are not encrypted. * `pg_createsubscriber` is not supported. -* Temporary files created when queries exceed `work_mem` are not encrypted. These files may persist if the query runs for a long time or the server crashes which can expose sensitive data in plaintext on disk. +* Temporary files created when queries exceed `work_mem` are not encrypted. These files may persist during long-running queries or after a server crash which can expose sensitive data in plaintext on disk. -## WAL tool compatibility (limited support) +## WAL tool compatibility The following tools and extensions in Percona Distribution for PostgreSQL have been tested and verified to work with `pg_tde` WAL encryption: -??? note "Click to expand" - * Patroni - * `pg_basebackup` (with `--wal-method=stream` or `--wal-method=none`), for details on using `pg_basebackup` with WAL encryption, see [Backup with WAL encryption enabled](../how-to/backup-wal-enabled.md) - * `pg_resetwal` - * `pg_rewind` - * `pg_upgrade` - * `pg_waldump` - * pgBackRest +* Patroni, for an example configuration see the following [Patroni configuration file](#example-patroni-configuration) +* `pg_basebackup` (with `--wal-method=stream` or `--wal-method=none`), for details on using `pg_basebackup` with WAL encryption, see [Backup with WAL encryption enabled](../how-to/backup-wal-enabled.md) +* `pg_resetwal` +* `pg_rewind` +* `pg_upgrade` +* `pg_waldump` +* pgBackRest + +## Example Patroni configuration + +The following is a Percona-tested example configuration. + +??? example "Click to expand the Percona-tested Patroni configuration" + ```yaml + # Example Patroni configuration file maintained by Percona + # Source: https://github.com/jobinau/pgscripts/blob/main/patroni/patroni.yml + scope: postgres + namespace: /db/ + name: postgresql0 + + restapi: + listen: 0.0.0.0:8008 + connect_address: 127.0.0.1:8008 + + etcd: + host: 127.0.0.1:2379 + + bootstrap: + dcs: + ttl: 30 + loop_wait: 10 + retry_timeout: 10 + maximum_lag_on_failover: 1048576 + postgresql: + use_pg_rewind: true + use_slots: true + parameters: + max_connections: 100 + shared_buffers: 1GB + wal_level: replica + hot_standby: "on" + wal_keep_size: 256MB + max_wal_senders: 10 + max_replication_slots: 10 + + initdb: + - encoding: UTF8 + - data-checksums + + pg_hba: + - host replication replicator 127.0.0.1/32 md5 + - host all all 0.0.0.0/0 md5 + + postgresql: + listen: 0.0.0.0:5432 + connect_address: 127.0.0.1:5432 + data_dir: /var/lib/postgresql/data + bin_dir: /usr/lib/postgresql/14/bin + authentication: + replication: + username: replicator + password: rep-pass + superuser: + username: postgres + password: secretpassword + ``` + +!!! warning + The above example is Percona-tested, but Patroni versions differ, especially with discovery backends such as `etcd`. Ensure you adjust the configuration to match your environment, version, and security requirements. ## Next steps From 58e53e893da201b1ade762759255653adfa80204 Mon Sep 17 00:00:00 2001 From: Dragos Andriciuc Date: Thu, 21 Aug 2025 16:49:46 +0300 Subject: [PATCH 14/15] add a new paragraph for unsupported tools under WAL tool compat and remove from limitations --- contrib/pg_tde/documentation/docs/index/tde-limitations.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/contrib/pg_tde/documentation/docs/index/tde-limitations.md b/contrib/pg_tde/documentation/docs/index/tde-limitations.md index b988347f028fa..d80f8b07db76b 100644 --- a/contrib/pg_tde/documentation/docs/index/tde-limitations.md +++ b/contrib/pg_tde/documentation/docs/index/tde-limitations.md @@ -3,11 +3,15 @@ Limitations of `pg_tde` {{release}}: * PostgreSQL’s internal system tables, which include statistics and metadata, are not encrypted. -* `pg_createsubscriber` is not supported. * Temporary files created when queries exceed `work_mem` are not encrypted. These files may persist during long-running queries or after a server crash which can expose sensitive data in plaintext on disk. ## WAL tool compatibility +The following tools and extensions in Percona Distribution for PostgreSQL are not currently supported: + +* `pg_createsubscriber` +* `pg_verifybackup` (checksum mismatch with encrypted WAL) + The following tools and extensions in Percona Distribution for PostgreSQL have been tested and verified to work with `pg_tde` WAL encryption: * Patroni, for an example configuration see the following [Patroni configuration file](#example-patroni-configuration) From b84c6c4820fcb4e105fecc3f8afdc071fe420785 Mon Sep 17 00:00:00 2001 From: Dragos Andriciuc Date: Fri, 22 Aug 2025 12:22:42 +0300 Subject: [PATCH 15/15] improve supported vs unsupported tools section - removed approved from overview for key providers --- .../docs/global-key-provider-configuration/overview.md | 2 +- .../pg_tde/documentation/docs/index/tde-limitations.md | 8 ++++++-- 2 files changed, 7 insertions(+), 3 deletions(-) diff --git a/contrib/pg_tde/documentation/docs/global-key-provider-configuration/overview.md b/contrib/pg_tde/documentation/docs/global-key-provider-configuration/overview.md index 0a43179fde400..14aa13e569c6b 100644 --- a/contrib/pg_tde/documentation/docs/global-key-provider-configuration/overview.md +++ b/contrib/pg_tde/documentation/docs/global-key-provider-configuration/overview.md @@ -11,7 +11,7 @@ To use an external KMS with `pg_tde`, follow these two steps: !!! note While key files may be acceptable for **local** or **testing environments**, KMS integration is the recommended approach for production deployments. -`pg_tde` has been tested and approved with the following key providers: +`pg_tde` has been tested with the following key providers: | KMS Provider | Description | Documentation | |--------------------|-------------------------------------------------------|---------------| diff --git a/contrib/pg_tde/documentation/docs/index/tde-limitations.md b/contrib/pg_tde/documentation/docs/index/tde-limitations.md index d80f8b07db76b..5b4e432872220 100644 --- a/contrib/pg_tde/documentation/docs/index/tde-limitations.md +++ b/contrib/pg_tde/documentation/docs/index/tde-limitations.md @@ -5,15 +5,19 @@ Limitations of `pg_tde` {{release}}: * PostgreSQL’s internal system tables, which include statistics and metadata, are not encrypted. * Temporary files created when queries exceed `work_mem` are not encrypted. These files may persist during long-running queries or after a server crash which can expose sensitive data in plaintext on disk. -## WAL tool compatibility +## Currently unsupported WAL tools -The following tools and extensions in Percona Distribution for PostgreSQL are not currently supported: +The following tools are currently unsupported with `pg_tde` WAL encryption: * `pg_createsubscriber` * `pg_verifybackup` (checksum mismatch with encrypted WAL) The following tools and extensions in Percona Distribution for PostgreSQL have been tested and verified to work with `pg_tde` WAL encryption: +## Supported WAL tools + +The following tools have been tested and verified by Percona to work with `pg_tde` WAL encryption: + * Patroni, for an example configuration see the following [Patroni configuration file](#example-patroni-configuration) * `pg_basebackup` (with `--wal-method=stream` or `--wal-method=none`), for details on using `pg_basebackup` with WAL encryption, see [Backup with WAL encryption enabled](../how-to/backup-wal-enabled.md) * `pg_resetwal`