From e376d34464338f9906a02122d4d53b10a5257d29 Mon Sep 17 00:00:00 2001 From: Dragos Andriciuc Date: Wed, 18 Jun 2025 15:18:15 +0300 Subject: [PATCH 1/6] Update multi-tenant-setup.md updated SELECT parameters Added that KMIP server setup is out of scope. Added a link to KMS configuration chapter in the intro to Key provider config --- .../docs/how-to/multi-tenant-setup.md | 71 +++++++++++++++---- 1 file changed, 58 insertions(+), 13 deletions(-) diff --git a/contrib/pg_tde/documentation/docs/how-to/multi-tenant-setup.md b/contrib/pg_tde/documentation/docs/how-to/multi-tenant-setup.md index 247a1878c254b..5b9525ff760d7 100644 --- a/contrib/pg_tde/documentation/docs/how-to/multi-tenant-setup.md +++ b/contrib/pg_tde/documentation/docs/how-to/multi-tenant-setup.md @@ -50,18 +50,27 @@ Load the `pg_tde` at startup time. The extension requires additional shared memo ## Key provider configuration -You must do these steps for every database where you have created the extension. +You must do these steps for every database where you have created the extension. For more information on configurations, please see the [Configure Key Management (KMS)](../global-key-provider-configuration/index.md) topic. 1. Set up a key provider. === "With KMIP server" + The KMIP server setup is out of scope of this document. + Make sure you have obtained the root certificate for the KMIP server and the keypair for the client. The client key needs permissions to create / read keys on the server. Find the [configuration guidelines for the HashiCorp Vault Enterprise KMIP Secrets Engine](https://developer.hashicorp.com/vault/tutorials/enterprise/kmip-engine). For testing purposes, you can use the PyKMIP server which enables you to set up required certificates. To use a real KMIP server, make sure to obtain the valid certificates issued by the key management appliance. ```sql - SELECT pg_tde_add_database_key_provider_kmip('provider-name','kmip-addr', 5696, '/path_to/client_cert.pem', '/path_to/client_key.pem', '/path_to/server_certificate.pem'); + SELECT pg_tde_add_database_key_provider_kmip( + 'provider-name', + 'kmip-addr', + 5696, + '/path_to/client_cert.pem', + '/path_to/client_key.pem', + '/path_to/server_certificate.pem' + ); ``` where: @@ -76,7 +85,14 @@ You must do these steps for every database where you have created the extension. :material-information: Warning: This example is for testing purposes only: ```sql - SELECT pg_tde_add_database_key_provider_kmip('kmip', '127.0.0.1', 5696, '/tmp/client_cert_jane_doe.pem', '/tmp/client_key_jane_doe.pem', '/tmp/server_certificate.pem'); + SELECT pg_tde_add_database_key_provider_kmip( + 'kmip', + '127.0.0.1', + 5696, + '/tmp/client_cert_jane_doe.pem', + '/tmp/client_key_jane_doe.pem', + '/tmp/server_certificate.pem' + ); ``` === "With HashiCorp Vault" @@ -84,7 +100,12 @@ You must do these steps for every database where you have created the extension. The Vault server setup is out of scope of this document. ```sql - SELECT pg_tde_add_database_key_provider_vault_v2('provider-name', 'url', 'mount', 'secret_token_path', 'ca_path'); + SELECT pg_tde_add_database_key_provider_vault_v2( + 'provider-name', + 'url', 'mount', + 'secret_token_path', + 'ca_path' + ); ``` where: @@ -96,8 +117,14 @@ You must do these steps for every database where you have created the extension. :material-information: Warning: This example is for testing purposes only: - ``` - SELECT pg_tde_add_database_key_provider_file_vault_v2('my-vault','http://vault.vault.svc.cluster.local:8200,'secret/data','hvs.zPuyktykA...example...ewUEnIRVaKoBzs2', NULL); + ```sql + SELECT pg_tde_add_database_key_provider_file_vault_v2( + 'my-vault', + 'http://vault.vault.svc.cluster.local:8200', + 'secret/data', + 'hvs.zPuyktykA...example...ewUEnIRVaKoBzs2', + NULL + ); ``` === "With a keyring file (not recommended)" @@ -105,30 +132,42 @@ You must do these steps for every database where you have created the extension. This setup is intended for development and stores the keys unencrypted in the specified data file. ```sql - SELECT pg_tde_add_database_key_provider_file('provider-name', '/path/to/the/keyring/data.file'); + SELECT pg_tde_add_database_key_provider_file( + 'provider-name', + '/path/to/the/keyring/data.file' + ); ``` :material-information: Warning: This example is for testing purposes only: ```sql - SELECT pg_tde_add_database_key_provider_file('file-keyring', '/tmp/pg_tde_test_local_keyring.per'); + SELECT pg_tde_add_database_key_provider_file( + 'file-keyring', + '/tmp/pg_tde_test_local_keyring.per' + ); ``` 2. Create a key ```sql - SELECT pg_tde_create_key_using_database_key_provider('name-of-the-key', 'provider-name'); + SELECT pg_tde_create_key_using_database_key_provider( + 'name-of-the-key', + 'provider-name' + ); ``` where: * `name-of-the-key` is the name of the principal key. You will use this name to identify the key. - * `provider-name` is the name of the key provider you added before. The principal key will be associated with this provider. + * `provider-name` is the name of the key provider you added before. The principal key is associated with this provider and it is the location where it is stored and fetched from. :material-information: Warning: This example is for testing purposes only: ```sql - SELECT pg_tde_create_key_using_database_key_provider('test-db-master-key', 'file-vault'); + SELECT pg_tde_create_key_using_database_key_provider( + 'test-db-master-key', + 'file-vault' + ); ``` !!! note @@ -137,7 +176,10 @@ You must do these steps for every database where you have created the extension. 3. Use the key as principal key ```sql - SELECT pg_tde_set_key_using_database_key_provider('name-of-the-key', 'provider-name'); + SELECT pg_tde_set_key_using_database_key_provider( + 'name-of-the-key', + 'provider-name' + ); ``` where: @@ -148,5 +190,8 @@ You must do these steps for every database where you have created the extension. :material-information: Warning: This example is for testing purposes only: ```sql - SELECT pg_tde_set_key_using_database_key_provider('test-db-master-key','file-vault'); + SELECT pg_tde_set_key_using_database_key_provider( + 'test-db-master-key', + 'file-vault' + ); ``` From 7c0b59bab02d27ce80b85639d42f6aa32efc31ef Mon Sep 17 00:00:00 2001 From: Dragos Andriciuc Date: Thu, 19 Jun 2025 13:36:07 +0300 Subject: [PATCH 2/6] updated uninstall/table-access/ * Reorganized Uninstall, added warning note and added a new step to ensure user knows he needs to decrypt or drop encrypted tables * Updated table access ALTER SYSTEM command do be easier to read * updated limitations, removed rewind mention and added WAL note as text, made small changes to RC version. Added note for KMS, improved system tables text. --- .../documentation/docs/how-to/uninstall.md | 37 ++++++++++++------- .../docs/index/table-access-method.md | 6 +-- .../docs/index/tde-limitations.md | 9 ++--- 3 files changed, 31 insertions(+), 21 deletions(-) diff --git a/contrib/pg_tde/documentation/docs/how-to/uninstall.md b/contrib/pg_tde/documentation/docs/how-to/uninstall.md index 7901f778fd8f4..c5fff2beb53bd 100644 --- a/contrib/pg_tde/documentation/docs/how-to/uninstall.md +++ b/contrib/pg_tde/documentation/docs/how-to/uninstall.md @@ -1,34 +1,45 @@ # Uninstall pg_tde -If you no longer wish to use TDE in your deployment, you can remove the `pg_tde` extension. To do so, your user must have the superuser privileges, or a database owner privileges in case you only want to remove it from a single database. +If you no longer wish to use TDE in your deployment, you can remove the `pg_tde` extension. To do so, you must have superuser privileges, or database owner privileges in case you only want to remove it from a single database. -Here's how to do it: +!!! warning + This process removes the extension, but does not decrypt data automatically. Only uninstall the extension after all encrypted data **has been removed or decrypted**. -1. Drop the extension using the `DROP EXTENSION` command: +To uninstall `pg_tde`, follow these steps: + +1. Decrypt or drop encrypted tables: + + Before removing the extension, you must either **decrypt** or **drop** all encrypted tables. `pg_tde` does not support decrypting tables in-place yet. Therefore: + + - To preserve the data: manually copy it into unencrypted tables + - To discard data: drop the encrypted tables + +2. Drop the extension using the `DROP EXTENSION` command: ```sql DROP EXTENSION pg_tde; ``` - This command will fail if there are still encrypted tables in the database. + Alternatively, to remove everything at once: - In this case, you must drop the dependent objects manually. Alternatively, you can run the `DROP EXTENSION ... CASCADE` command to drop all dependent objects automatically. + ```sql + DROP EXTENSION pg_tde CASCADE; + ``` - Note that the `DROP EXTENSION` command does not delete the `pg_tde` data files related to the database. + !!! note + The `DROP EXTENSION` command does not delete the underlying `pg_tde`-specific data files from disk. -2. Run the `DROP EXTENSION` command against every database where you have enabled the `pg_tde` extension, if the goal is to completely remove the extension. This also includes the template databases, in case `pg_tde` was previously enabled there. +3. Run the `DROP EXTENSION` command against every database where you have enabled the `pg_tde` extension, if the goal is to completely remove the extension. This also includes the template databases, in case `pg_tde` was previously enabled there. -3. Remove any reference to `pg_tde` GUC variables from the PostgreSQL configuration file. +4. Remove any reference to `pg_tde` GUC variables from the PostgreSQL configuration file. -4. Modify the `shared_preload_libraries` and remove the 'pg_tde' from it. Use the `ALTER SYSTEM` command for this purpose, or edit the configuration file. +5. Modify the `shared_preload_libraries` and remove the 'pg_tde' from it. Use the `ALTER SYSTEM` command for this purpose, or edit the configuration file. !!! warning - - Once `pg_tde` is removed from the `shared_preload_libraries`, reading any leftover encrypted files will fail. Removing the extension from the `shared_preload_libraries` is also possible if the extension is still installed in some databases. - + Once `pg_tde` is removed from the `shared_preload_libraries`, reading any leftover encrypted files will fail. Removing the extension from the `shared_preload_libraries` is also possible if the extension is still installed in some databases. Make sure to do this only if the server has no encrypted files in its data directory. -5. Start or restart the `postgresql` cluster to apply the changes. +6. Start or restart the `postgresql` cluster to apply the changes. * On Debian and Ubuntu: diff --git a/contrib/pg_tde/documentation/docs/index/table-access-method.md b/contrib/pg_tde/documentation/docs/index/table-access-method.md index 1e366f1f16887..b308cc1aedbcb 100644 --- a/contrib/pg_tde/documentation/docs/index/table-access-method.md +++ b/contrib/pg_tde/documentation/docs/index/table-access-method.md @@ -52,9 +52,9 @@ Here is how you can set the new default table access method: 1. Add the access method to the `default_table_access_method` parameter: - === "via the SQL statement" + === "via the `ALTER SYSTEM` command" - Use the `ALTER SYSTEM` command. This requires superuser or ALTER SYSTEM privileges. + Use `ALTER SYSTEM`, it requires superuser or `ALTER SYSTEM` privileges. ```sql ALTER SYSTEM SET default_table_access_method = tde_heap; @@ -68,7 +68,7 @@ Here is how you can set the new default table access method: default_table_access_method = 'tde_heap' ``` - === "via the SET command" + === "via the `SET` command" You can use the SET command to change the default table access method temporarily, for the current session. diff --git a/contrib/pg_tde/documentation/docs/index/tde-limitations.md b/contrib/pg_tde/documentation/docs/index/tde-limitations.md index cc10f80519b22..271665a0f17be 100644 --- a/contrib/pg_tde/documentation/docs/index/tde-limitations.md +++ b/contrib/pg_tde/documentation/docs/index/tde-limitations.md @@ -1,10 +1,9 @@ # Limitations of pg_tde -* Keys in the local keyfile are stored unencrypted. For better security we recommend using the Key management storage. -* System tables are currently not encrypted. This means that statistics data and database metadata are currently not encrypted. - -* `pg_rewind` doesn't work with encrypted WAL for now. We plan to fix it in future releases. -* `pg_tde` Release candidate is incompatible with `pg_tde`Beta2 due to significant changes in code. There is no direct upgrade flow from one version to another. You must [uninstall](../how-to/uninstall.md) `pg_tde` Beta2 first and then [install](../install.md) and configure the new Release Candidate version. +* Keys in the local keyfile are stored **unencrypted**. For better security we recommend using the [Key management storage](../global-key-provider-configuration/index.md). +* System tables, which include statistics data and database statistics, are currently **not encrypted**. +* The WAL encryption feature is currently in beta and is not effective unless explicitly enabled. It is not yet production ready. **Do not enable this feature in production environments**. +* `pg_tde` RC 2 is incompatible with `pg_tde` Beta2 due to significant changes in code. There is no direct upgrade flow from one version to another. You must [uninstall](../how-to/uninstall.md) `pg_tde` Beta2 first and then [install](../install.md) and configure the new Release Candidate version. !!! important This is the {{release}} version of the extension and **it is not meant for production use yet**. We encourage you to use it in testing environments and [provide your feedback](https://forums.percona.com/c/postgresql/pg-tde-transparent-data-encryption-tde/82). From 49abe96361218c37cfbdd3bd7fc4e1ab084e9e8a Mon Sep 17 00:00:00 2001 From: Dragos Andriciuc Date: Thu, 26 Jun 2025 12:19:30 +0300 Subject: [PATCH 3/6] updated decrypt table command and removed preserve data mention for step 1 --- contrib/pg_tde/documentation/docs/how-to/uninstall.md | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/contrib/pg_tde/documentation/docs/how-to/uninstall.md b/contrib/pg_tde/documentation/docs/how-to/uninstall.md index c5fff2beb53bd..67630287e464d 100644 --- a/contrib/pg_tde/documentation/docs/how-to/uninstall.md +++ b/contrib/pg_tde/documentation/docs/how-to/uninstall.md @@ -9,9 +9,14 @@ To uninstall `pg_tde`, follow these steps: 1. Decrypt or drop encrypted tables: - Before removing the extension, you must either **decrypt** or **drop** all encrypted tables. `pg_tde` does not support decrypting tables in-place yet. Therefore: + Before removing the extension, you must either **decrypt** or **drop** all encrypted tables: + + - To decrypt a table, run: + + ```sql + alter table set access method heap; + ``` - - To preserve the data: manually copy it into unencrypted tables - To discard data: drop the encrypted tables 2. Drop the extension using the `DROP EXTENSION` command: From 12a825e0c8a9b672dbb15b3eb63096edb6428d52 Mon Sep 17 00:00:00 2001 From: Dragos Andriciuc Date: Thu, 26 Jun 2025 12:24:20 +0300 Subject: [PATCH 4/6] minor text fix for step 1 minor text fix. --- contrib/pg_tde/documentation/docs/how-to/uninstall.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/contrib/pg_tde/documentation/docs/how-to/uninstall.md b/contrib/pg_tde/documentation/docs/how-to/uninstall.md index 67630287e464d..b23b6a063a3b6 100644 --- a/contrib/pg_tde/documentation/docs/how-to/uninstall.md +++ b/contrib/pg_tde/documentation/docs/how-to/uninstall.md @@ -17,7 +17,7 @@ To uninstall `pg_tde`, follow these steps: alter table
set access method heap; ``` - - To discard data: drop the encrypted tables + - To discard data, drop the encrypted tables. 2. Drop the extension using the `DROP EXTENSION` command: From c4548583d620f42b7e4aa7678c8453272373fe21 Mon Sep 17 00:00:00 2001 From: Dragos Andriciuc Date: Thu, 26 Jun 2025 13:17:48 +0300 Subject: [PATCH 5/6] small param update small param update --- contrib/pg_tde/documentation/docs/how-to/uninstall.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/contrib/pg_tde/documentation/docs/how-to/uninstall.md b/contrib/pg_tde/documentation/docs/how-to/uninstall.md index b23b6a063a3b6..16a97e179a86e 100644 --- a/contrib/pg_tde/documentation/docs/how-to/uninstall.md +++ b/contrib/pg_tde/documentation/docs/how-to/uninstall.md @@ -14,7 +14,7 @@ To uninstall `pg_tde`, follow these steps: - To decrypt a table, run: ```sql - alter table
set access method heap; + ALTER TABLE
SET ACCESS METHOD heap; ``` - To discard data, drop the encrypted tables. From 18b8916c5c70ed43df3dc0f7f81c03b0062b56ef Mon Sep 17 00:00:00 2001 From: Dragos Andriciuc Date: Thu, 26 Jun 2025 14:16:19 +0300 Subject: [PATCH 6/6] new line for pg_tde_add_database_key_provider_vault_v2 --- contrib/pg_tde/documentation/docs/how-to/multi-tenant-setup.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/contrib/pg_tde/documentation/docs/how-to/multi-tenant-setup.md b/contrib/pg_tde/documentation/docs/how-to/multi-tenant-setup.md index 5b9525ff760d7..19a9974b3df28 100644 --- a/contrib/pg_tde/documentation/docs/how-to/multi-tenant-setup.md +++ b/contrib/pg_tde/documentation/docs/how-to/multi-tenant-setup.md @@ -102,7 +102,8 @@ You must do these steps for every database where you have created the extension. ```sql SELECT pg_tde_add_database_key_provider_vault_v2( 'provider-name', - 'url', 'mount', + 'url', + 'mount', 'secret_token_path', 'ca_path' );